System and method for micro-assembly control using machine learning enabled by digital computers

Abstract

The present invention describes systems and methods that allow a micro-object to be moved to a desired position using a machine learning algorithm. A sensor such as a high-speed camera or capacitive sensing tracks the position of the object. A dynamic potential energy landscape for manipulating the object is generated by controlling each of the electrodes in an electrode array. One or more computing devices are used to: estimate the initial position of the micro-object using a sensor; generate a continuous representation of a dynamic model for the movement of the micro-object due to electrode potentials generated by at least some of the electrodes, and use automatic differentiation and Gaussian quadrature rule on the dynamic model to derive an optimized potential generated by the electrode to move the micro-object to a desired position; and map the calculated optimized electrode potential to the array to activate the electrode.

Description

[0001] This invention was made with government support under Contract No. HR00111990027 awarded by the Defense Advanced Research Projects Agency. The U.S. Government has certain rights in this invention. Technical Field

[0002] The present patent application relates generally to microassembly control, and in particular, to systems and methods for microassembly control that enable machine learning with a digital computer. Background Art

[0003] Micro- and nano-scale particle manipulation has attracted a lot of research interest. The degree to which the assembly of micro-objects (objects whose sizes are measured in micrometers) and nano-objects (objects whose sizes are measured in nanometers) can be controlled can make a significant difference in many technologies, including in micro-manufacturing, biology, and medicine. For example, the manufacture of reconfigurable circuits can be improved by being able to accurately control the position of micro-objects (such as capacitors and resistors) to manufacture circuits with desired performance. Similarly, the production of photovoltaic solar arrays can benefit from being able to place photovoltaic cells with certain qualities in specific locations on the array. Such batteries may be too small to achieve the desired placement of batteries via human or robot manipulation, and another transport mechanism is required. The micro- and nano-assembly of particles can also be used to design the microstructure of materials, such as biological cells assembled into tissues. There are many other technical fields in which increasing the control of the assembly of micro- and nano-objects can provide significant benefits.

[0004] Existing technologies do not allow the movement of micro- and nano-objects to be controlled with the desired degree of precision. For example, uncontrolled mechanical stirring is often used for directed particle assembly. However, this technique cannot achieve the near 100% yield necessary for certain industrial applications, such as electronics assembly.

[0005] Previous work has also attempted to use electric fields to orient the movement of micro-objects. For example, in Matei et al., "Micro-scale chiplets position control", Journal of Microelectromechanical Systems, Vol. 28, No. 4, pp. 643-655, August 2019, and Matei et al., "Towards printing as an electronics manufacturing method: Micro-scale chiplet position control", 2017 American Control Conference (ACC), pp. 1549-1555, May 2017, the disclosures of which are incorporated by reference, disclose a micro-object control strategy based on a one-step model predictive control method. The disclosed one-dimensional model is based on capacitance, but the actuation mechanism used is based on spiral electrodes, which limits the number of simultaneously actuated electrodes that can be used by the model. In addition, it is assumed that the size of the particles whose movement is controlled is small enough not to interfere with the electric field, which can limit the applicability of the technology to larger particles.

[0006] Similarly, Wang et al., "Dielectrophoretic manipulation of cells with spiral electrodes," Biophysical Journal, Vol. 72, No. 4, pp. 1887-1899, 199, describes studies investigating the effects of dielectrophoresis on cancer cells. This work assumes that the particles involved are spherical and small enough that the electric field is not affected by their presence, limiting the applicability of the technique.

[0007] Control schemes for individual and integrated control of colloids are described in Tara D. Edwards and Michael A. Bevan, "Controlling colloidal particles with electric fields", Langmuir, Vol. 30, No. 36, pp. 10793–10803, 2014, PMID: 24601635 ("Edwards"), the disclosure of which is incorporated by reference. Specifically, Edwards demonstrated how to manipulate individual and integrated colloidal particles (1 μm to 3 μm diameter) in water and sodium hydroxide solutions by electrophoresis and electroosmosis using non-uniform electric fields. The relative size of the colloids to the electrodes used to generate the fields, the medium in which the particles are immersed, and the resulting mathematical models do not allow the use of the techniques in certain industrial applications. In particular, the techniques are not suitable for assembling micro-objects that are even slightly larger than those discussed in the Edwards paper. In addition, the control schemes used involve high frequency signals (MHz), which further limits the applicability of such techniques.

[0008] Similarly, Qian et al., "On-demand and location selective particle assembly via electrophoretic deposition for fabricating structures with particle-to-particle precision," Langmuir, Vol. 31, No. 12, pp. 3563-3568, 2015, PMID: 25314133, the disclosure of which is incorporated by reference, demonstrates single particle precision and location selective particle deposition, where electrophoretic forces are the primary driver for particle (2 μm polystyrene beads) manipulation. The control scheme employed is based on building large energy wells near the desired locations of the nanoparticles. However, the technique does not allow for adequate sorting and placement of individual objects. Furthermore, the technique does not allow for adequate manipulation of asymmetric objects, such as semiconductor chips, which require the use of directional control to build electronic systems.

[0009] Several works, such as Xue et al., "Optimal design of a colloidal self-assembly process", IEEE Transactions on Control Systems Technology, Vol. 22, No. 5, pp. 1956-1963, September 2014, and Xue et al., "Mdp based optimal control for a colloidal self-assembly system", American Control Conference (ACC), 2013, pp. 3397-3402, June 2013, the disclosures of which are incorporated by reference, describe controlling a random colloidal assembly process that drives the system to a desired high crystallinity state and is based on a Markov decision process optimal control strategy. The dynamic model is based on the Langevin equation parameterized by the actuator. However, in these works, individual particles are not directly manipulated, and it is unclear how this approach can be used when assembling circuits. In addition, the size of the particles used (≈3 μm in diameter) is so small that they have little interference with the electric field fully shaped by the actuation potential. In addition, the time scale for achieving the desired state will make it challenging to achieve the goal of high throughput using this approach.

[0010] Other self-assembly control methods are described by Grzelczak et al., “Directed self-assembly of nanoparticles,” ACS Nano, Vol. 4, No. 7, pp. 3591-3605, 2010, PMID: 20568710; Paulson et al., “Control of self-assembly in micro-and nano-scale systems,” Journal of Process Control, Vol. 27, pp. 38-49, 2015; Mastrangeli et al., “Automated real-time control of fluidic self-assembly of microparticles,” Robotics and Automation (ICRA), 2014 IEEE International Conference on Automation, pp. 5860-5865, May 2014; and Paulson et al., “Control of self-assembly in micro-and nano-scale systems,” Journal of Process Control, Vol. 27, pp. 38-49, 2015. Control, Vol. 27, pp. 38-49, 2015, the disclosures of which are incorporated by reference, do not themselves allow for easy scaling of the number of objects being moved.

[0011] A water-based solution in which the particles are immersed is a common choice of contrast medium, such as described by Edman et al., "Electric field directed assembly of an InGaAs LED onto silicon circuitry", IEEE Photonics Technology Letters, Vol. 12, No. 9, pp. 1198-1200, September 2000, and Tolley et al., "Dynamically programmable fluidic assembly", Applied Physics Letters, Vol. 93, No. 25, 2008, the disclosures of which are incorporated by reference. In this case, both electrophoretic forces and fluid motion of electroosmotic flow are used to drive the particles. However, water does not behave like a dielectric and therefore cannot generate an electric field, resulting in a lower force for moving objects, significantly limiting the size of objects that can be moved using this setup.

[0012] Precise control of cells, quantum dots, and nanowires based on electroosmosis is used in the following documents: Mathai et al., "Simultaneous positioning and orientation of single nano-wires using flow control", RSC Adv., Vol. 3, pp. 2677-2682, 2013; and Probst et al., "Flow control of small objects on chip: Manipulating live cells, quantum dots, and nanowires", IEEE Control Systems, Vol. 32, No. 2, pp. 26-53, April 2012, the disclosures of which are incorporated by reference. The authors use a linear model of the electrode potential, and the effect of the particles on the electric field distribution is negligible. However, when the driving force is mainly dielectrophoresis, the linearity presented in the electrode potential does not hold, thus limiting the applicability of these techniques. In addition, the linearity presented may not be applicable to objects on the microscale.

[0013] Finally, Zemánek et al., “Feedback-controlled dielectrophoretic micromanipulation,” 2018 International Conference on Manipulation, Automation and Robotics at Small Scales (MARSS), July 2018, the disclosure of which is incorporated by reference, describes dielectrophoretic-based feedback for microsphere manipulation. The authors used a simulated annealing approach to solve the optimal control problem, where they exploited the spherical shape of the particles when modeling the system. However, the reliance of this technique on the spherical shape of the particles being moved limits the applicability of the technique.

[0014] Therefore, there is a need for a scalable way to control the assembly of micro-objects of various shapes that can be achieved in real time. Summary of the invention

[0015] The present invention describes systems and methods for moving micro-objects to desired locations using machine learning algorithms. Sensors such as high-speed cameras or capacitive sensor arrays track the location of the object. The programmable electrode array can be individually controlled with integrated electronics (such as an active matrix array) or with a video projector that projects a light pattern that addresses a light switch or phototransistor to dynamically control each electrode and thereby generate a dynamic potential energy landscape for manipulating the object. Dielectrophoretic (DEP) forces and electrophoretic (EP) forces are example forces that can be generated to move micro-objects. The micro-object can be a semiconductor chip, an integrated circuit, a particle, a nanodevice or structure, or any object that can be physically manipulated by an electrode array. One or more computing devices are used to: process sensing data, such as an image captured by a camera, to estimate the initial position of the micro-object; generate a continuous representation of a dynamic model for the movement of the micro-object due to electrode potentials generated by at least some of the electrodes, and use automatic differentiation and Gaussian quadrature rule on the dynamic model to derive an optimized potential generated by the electrode to move the micro-object to a desired location; and map the calculated new electrode potential to the electrode array. In the case of an electrode array with integrated active matrix addressing electronics, the potential on each electrode of the array is updated electronically. In the case of an electrode array controlled by illuminating an array of phototransistors or photoswitches connected to each electrode with a video projector, the potential on each electrode is updated by updating an image projected onto the array using the video projector. The projected image activates or deactivates the electrode as indicated by the control input. The system and method allow real-time control of micro-object positioning in a scalable and computationally efficient manner.

[0016] In one embodiment, a system and method for micro-assembly control enabled by machine learning with a digital computer are provided. The system includes one or more processors, the one or more processors implementing one or more machine learning algorithms, the one or more processors being configured to: obtain one or more parameters of a system for positioning a micro-object, the system including a plurality of electrodes, each of which is programmable, the electrodes being configured to cause movement of the micro-object when one or more potentials are generated by one or more of the electrodes when the micro-object is close to the electrodes; modeling the capacitance between the micro-object and the electrodes using the parameters; estimating the position of the micro-object with a sensor; receiving another position of the micro-object; defining a continuous representation of a dynamic model of movement of the micro-object caused by the electrode potential generated by at least some of the electrodes using the capacitance as a result of the electrode potential generated by at least some of the electrodes, wherein the dynamic model is associated with constraints represented as desired; applying one of the machine learning algorithms to at least a portion of the continuous representation of the dynamic model to perform optimization of the electrode potential generated by at least some of the electrodes to move the micro-object from the estimated position to another position; using the optimized electrode potential to actuate at least some of the electrodes to cause movement of the micro-object from the estimated position to another position.

[0017] Other embodiments of the present invention will become apparent to those skilled in the art from the following detailed description, wherein embodiments of the present invention are described by way of illustration of the best mode contemplated for carrying out the present invention. As will be appreciated, the present invention is capable of other and different embodiments, and its several details are capable of modification in various obvious respects, all without departing from the spirit and scope of the present invention. Therefore, the drawings and detailed description are to be regarded as illustrative rather than restrictive in nature. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a block diagram illustrating a system for enabling machine learning for micro-assembly control with a digital computer.

[0019] Figure 2 It is shown that according to one embodiment Figure 1 Diagram of the interaction between a video projector and a phototransistor array.

[0020] FIG. 3A to FIG. 3C According to one embodiment, Figure 1 Several views of examples of phototransistor arrays.

[0021] Figure 4 (A) to Figure 4 (B) are examples of using Figure 1 Examples of micro-object patterns achieved with the system.

[0022] Figure 5 It is shown that according to one embodiment Figure 1Diagram of the capacitive-based interaction between a micro-object and five electrodes of the system.

[0023] Figure 6 is a diagram showing a COMSOL model of two conductors (spherical micro-object and electrode) for capacitance calculation according to one embodiment.

[0024] Figure 7 A capacitance function equipped with training data generated by COMSOL simulations is shown according to one embodiment.

[0025] Figure 8 is a graph showing a 2D capacitance function determined from a ID model using symmetry according to one embodiment.

[0026] Fig. 9 is a diagram showing a control strategy obtained by solving equation (2) according to one embodiment.

[0027] Fig.10 is a diagram illustrating a control strategy for moving a micro-object to a desired position in two dimensions according to one embodiment.

[0028] Fig.11 is a flow chart illustrating a method for enabling machine learning for micro-assembly control with a digital computer.

[0029] Fig.12 is a diagram showing a method for using a Fig.11 Flowchart of the routine for modeling micro-object motion using capacitance used in the method.

[0030] Fig.13 is a diagram illustrating a dual mode control strategy according to one embodiment, where x is the micro-object position and the voltage polarity changes at x+b.

[0031] Fig.14 is a graph showing the force distribution as a function of b for Vmax=1 and σ=0.1.

[0032] Fig.15 is a diagram illustrating a dual mode control strategy according to one embodiment, where x is the micro-object position and the potential starts to become non-zero at x+b.

[0033] Fig.16 is a graph showing the force distribution as a function of b for V_max=1 and σ=0.1 using the control strategy of equation (6). DETAILED DESCRIPTION

[0034] The system and method described below provides an efficient and scalable way to find control strategies, electrode potential distributions that move micro-objects to desired locations. The analysis involves converting a micro-model (a dynamic model with distributed parameters) to a macro-model (a dynamic model with competitive distributions) and using machine learning-based optimization to find the control strategy. Converting the micro-model to a macro-model significantly increases the efficiency of the computation. In addition, the control strategy can be in one or two dimensions, where rotational transformations allow the use of a one-dimensional control strategy to derive the optimal two-dimensional control strategy.

[0035] Figure 1 is a block diagram of a system 10 illustrating micro-assembly control enabled by machine learning with the aid of a digital computer. The system 10 allows for coordinated assembly involving multiple objects 11, such as micro-objects. The size of the objects 11 ranges between the nanoscale (<1 μm) and the microscale (up to 100 μm), although other sizes are possible. The objects 11 can be spherical, but other shapes are possible, such as a rectangular shape, although other shapes are possible. In one embodiment, the diameter of the spherical object 11 can be 20 μm to 50 μm, although other diameters are possible. In one embodiment, the dimensions of the rectangular object 11 can be 200 μm×300 μm×100 μm, although other sizes are possible. The objects 11 are immersed in a dielectric fluid (not shown) contained in a housing (not shown), wherein the membrane layer 38 is lower than the dielectric fluid contained in the housing. In one embodiment, the dielectric fluid is The dielectric fluid 17 is made of a dielectric material such as aluminum oxide (AlOx), a semiconductor chip, an integrated circuit, a particle, a nano device or a structure, or any object that an electrode array can physically manipulate. In the description below, object 11 is referred to as micro-object 11, although other ways of naming object 11 are also possible. In one embodiment, the film can be a perfluoroalkoxy (PFA) film of 50 μm thick, although other types of films of other thicknesses are also possible.

[0036] Below the suspended micro-object 11 are a plurality of programmable electrodes 12 forming an array 35, the electrodes 12 being configured to generate a dynamic potential energy landscape for manipulating the object using dielectrophoretic ("DEP") forces and electrophoretic ("EP") forces. A membrane is laminated onto the electrodes 12. In one embodiment, the electrodes can be square in shape and made of copper, although other shapes and materials are possible. In one embodiment, the dimensions of the square electrodes 12 can be 16 μm in width and 100 μm in thickness, although other dimensions are possible in other embodiments. The array 35 may include multiple rows of electrodes 12, wherein each row includes multiple electrodes 12.

[0037] The potential generated by the electrode 12 is controlled by an array 13 of phototransistors, which includes an active matrix phototransistor backplane disposed on glass. The plurality of phototransistors on the backplane form an array 13, wherein each of the phototransistors in the array 13 controls the potential generated by a single electrode 12. Specifically, each of the phototransistors in the array 13 is attached to one electrode 12. The array 13 of phototransistors may have additional characteristics, such as those described in the following document: Rupp et al., "Chiplet micro-assembly printer", 2019 IEEE 69th Electronic Components and Technology Conference (ECTC), pp. 1312-1315, May 2019, the disclosure of which is incorporated by reference.

[0038] The array 13 is optically addressed by a video projector 14 to enable simultaneous dynamic control of the electrostatic energy potential and manipulation of the positions of multiple micro-objects 11. Specifically, the video projector 14 is used to address each phototransistor-controlled electrode 12, such as Figure 2 As shown, this allows easy scaling factor changes and stitching into larger arrays. Figure 2 It is shown that according to one embodiment Figure 1 FIG. 1 is a diagram of the interaction between a video projector 14 and a phototransistor array 13. The video projector 14 actuates the electrodes 12 by projecting a predefined pattern: an image 37 generated based on a control input: an electrical potential generated by the electrodes to achieve a desired motion of the micro-object 11, as further described below. The pixelated light constituting the image projected by the projector 14 charges each individual phototransistor 13 in the array to the necessary degree so that the electrode 12 charged by the phototransistor generates the desired electrical potential.

[0039] 3(A) to 3(C) show a schematic diagram of a Figure 1Several views of examples of phototransistor array 13 of. The phototransistor array 13 shown in reference figure 3 (A) to Figure 3 (B) is constructed using a manufacturing technology similar to that used to construct a display transistor backplane, so that the manufacturing technology is compatible with a large area (measurement instrument). Voltages up to + / -200V are controlled by these phototransistors to display a dynamically changing complex potential landscape. A single voltage signal is synchronized with optical addressing to charge the storage capacitor on each pixel. Dynamic electric field patterns are generated by refreshing the image pattern (control input) of the projection. An array 13 with 50 μm, 10 μm or even 3 μm spacing has been constructed, where the spacing is the distance between the centers of the electrodes to which the phototransistors are attached. Dielectric fluid 17 medium (isopar M) and various additives are used to control the charge.

[0040] return Figure 1 The system 10 also includes a high-speed camera 15 that tracks the position of the moved micro-object 11. Both the video projector 14 and the camera 15 are interfaced to one or more computing devices 16, which can control the electrodes 12 through the projector 14 to cause the formation of the desired micro-object 11 pattern. Figure 1 Examples of micro-object 11 patterns implemented by the system 10 of FIG. Other micro-object 11 patterns are also possible. In other embodiments, another sensor (such as using capacitive sensing) can be used to measure and track the position of the micro-object. In other embodiments, the electrode array 35 can have an integrated active matrix addressing electronics, and the potential on each electrode of the array is electronically updated when the electrode potential necessary for the desired micro-object movement has been generated.

[0041] The connection between one or more computing devices 16 can be direct, such as via a wired or wireless connection to physically close one or more computing devices 16, or the connection can be physically remote one or more computing devices 16 via an interconnected network, such as via the Internet or a cellular network. One or more computing devices 16 include one or more processors. In one embodiment, the computer processor can be a central processing unit (CPU), a graphics processing unit (GPU), or a mixture of CPUs and GPUs, although other types of processors or mixtures of processors are also possible. In one embodiment, two or more of the processors can perform processing in parallel, as further described in the following patents: U.S. Patent Application Serial No. 17 / 098,816, filed on November 16, 2020, entitled "System and Method For Multi-Object Micro-Assembly Control With The Aid of A Digital Computer", Matei et al., the disclosure of which is incorporated by reference. Additional efficiency and improved scalability can be obtained by parallel computing capabilities for tracking multiple particles and for voltage pattern generation.

[0042] Although one or more computing devices 16 are shown as servers, other types of computer devices are also possible. Computing device 16 may include one or more modules for executing the embodiments disclosed herein. These modules may be implemented as computer programs or processes written as source code in a conventional programming language and presented to a processor for execution as a target or byte code. Alternatively, these modules may also be implemented in hardware, as an integrated circuit or burned into a read-only memory component, and each of the computing devices 16 may act as a dedicated computer. For example, when these modules are implemented as hardware, the specific hardware is dedicated to performing the above-mentioned calculations and communications, and other computers cannot be used. In addition, when these modules are burned into a read-only memory component, the computer storing the read-only memory becomes dedicated to performing the operations described above that other computers cannot perform. Various specific implementations of source code and target and byte code may be maintained on a computer-readable storage medium, such as a floppy disk, a hard drive, a digital video disk (DVD), a random access memory (RAM), a read-only memory (ROM), and similar storage media. Other types of modules and module functions and other physical hardware components are also possible. For example, computing device 16 may include other components found in programmable computing devices, such as input / output ports, network interfaces, and non-volatile storage, but other components are also possible. In an embodiment where computing device 16 is a server, the server may also be cloud-based or a dedicated server.

[0043] One or more computing devices 16 interface with storage device 17 and execute dynamic modeler 18, which obtains parameters 19 of system 10, stores parameters 19 in storage device 17, and models the capacitance between electrode 12 and micro-object. Parameters 19 may include the diameter of micro-object 11, the size of electrode 12, dielectric fluid constant (e.g., ε=2), the fixed position of electrode 12, the material of micro-object 11 and electrode 12, and the vertical distance between micro-object 11 and electrode 12 (the "height" of micro-object 11). Other parameters 19 are still possible.

[0044] The design of the control strategy is based on a dynamic model of the motion of a micro-object under the action of the electric field generated by the electrode array. The result of applying an electric potential to the electrode 12 is the generation of dielectrophoretic and electrophoretic forces acting on the micro-object 11. The viscous drag proportional to the velocity opposes the motion of the micro-object (wherein in non-turbulent flow, i.e. when the Reynolds number is small, the drag is proportional to the velocity). Since the mass of the micro-object is negligible, the acceleration is neglected. Therefore, the dynamic model 20 for the motion of the micro-object can be described by the following:

[0045]

[0046]

[0047] where (x, y) represents the position of the micro-object 11 measured at the centroid of the chilplet, μ is the viscosity coefficient associated with the fluid, and F x (x,y) and F y (x,y) are the projections of the force caused by the potential field on the x-axis and y-axis respectively. x and F y is expressed as a function of the potential energy of the micro-object. The potential energy is calculated by using a capacitance-based circuit that aggregates the interaction between the electrode and the micro-object. Figure 5 It is shown that according to one embodiment Figure 1 Figure 1 shows the capacitive-based interaction between a micro-object 11 and five electrodes of a system. The micro-object and the electrodes act as metal plates; therefore, the capacitance of these capacitors depends on the position of the micro-object. Assume that U(x,y) represents the potential energy of the micro-object and the force F x and F y It can be formally expressed as and Potential energy can be expressed in terms of circuit capacity:

[0048]

[0049] Among them C i(x,y) is the capacitance between the micro-object and electrode i, V iis the potential of electrode i, V(x,y) is the potential of the micro-object assumed to be uniformly distributed over the entire micro-object. The integer N is the number of driving electrodes. By solving the reference Figure 5 The voltage and current in the circuit shown can be used to explicitly calculate the micro-object potential from the electrode potential. Specifically, the steady-state micro-object potential is given by:

[0050]

[0051] The feedback control design requires an explicit representation of the capacitance between the micro-object and the electrode. The capacitance modeler 18 uses COMSOL developed by COMSOL Inc. in Burlington, Massachusetts. Model 20 was created using high-fidelity simulation using COMSOL software ("COMSOL"), although other types of simulations using other types of software are possible. For symmetric micro-objects (e.g., beads), the capacitance is estimated by simulating a two-dimensional electrostatic COMSOL model assuming that the electrodes are also symmetric. This means that the capacitance function is of the form C i(η) =C(||η-η i ||), where η T = [x, y] represents the position of the micro-object, and is the fixed position of electrode i. Figure 6 is a diagram showing a COMSOL model of two conductors (spherical micro-object and electrode) for capacitance calculation according to one embodiment. Figure 5 , the electrode position is (0,0). Figure 6 As shown, in the COMSOL model, a copper plate with a width of 16 μm and a thickness of 100 nm and a spherical object of aluminum oxide (AlOx) with a diameter of 10 μm are surrounded by a dielectric with the same properties as the isopar-M solution. The capacitance matrix entries are calculated based on the charge generated on each conductor when an electric potential is applied to one of the conductors and the other conductor is set to ground. The COMSOL simulation does reflect the field distortion when the micro-object approaches the electrode. The parameters of the COMSOL electrostatic model include the diameter of the sphere, the electrode size, the dielectric fluid constant (ε = 2), and the position and material of the sphere and electrode. The height of the micro-object is fixed (z = 5 μm), and the position of the micro-object varies on the x-axis in the interval [-1 mm, 1 mm]. Due to the relationship between the size of the micro-object and the size of the electrode, the edge effect (electric field distortion at the edge) is significant. The capacitance function is parameterized using the error function: in is the error function, ξ is the distance between the center of the sphere and the center of the electrode assumed at the origin, and a i and c i >0 is a positive scalar, and δ is half the electrode spacing, i.e. 10 μm in this case.

[0052] Figure 7 A capacitance function equipped with training data generated by COMSOL simulations is shown according to one embodiment. Figure 7 The 1D model can be obtained by using The transformation is mapped to a 2D model, which results in the capacitance function For a spherical chiplet, considering only one of the parameterizations of the capacitance function, the resulting 2D shape is Figure 8 Depicted. Figure 8 is a graph showing a 2D capacitance function determined from a ID model using symmetry according to one embodiment.

[0053] The one or more computing devices 16 also execute a position tracker 36 that analyzes the image 21 captured by the high-speed camera 15 (or another sensor) to determine the position 22 of the micro-object 11. The position 22 and the dynamic capacitance model 20 are used to determine a control strategy 24 that includes the control input 27 (electric potential) required to be generated by the electrode 12 to guide the electrode to the desired position.

[0054] The determination of the control strategy 24 is performed using an electrode potential modeler 25 executed by one or more computing devices 16. As described above, the determination of the control strategy 24 involves converting a micromodel (a dynamic model 20 of specific distributed parameters) into a macromodel (a dynamic model 20 with a competitive distribution) and using machine learning-based optimization to find a control strategy. Converting the micromodel to a macromodel significantly increases the efficiency of the calculation. In addition, the control strategy 24 can be in one or two dimensions, where a rotation transformation allows the use of a one-dimensional control strategy to derive the best two-dimensional control strategy. To enable traceability, a mean field-like approximation of the dynamic model 20 is used, in which the discrete distribution of the array potential is not considered, but a continuous distribution is assumed. Under this assumption, a dual-mode control strategy, in which the voltage distribution moves polarity just in front of the micro-object and indeed guides the micro-object motion to the desired direction. The following description starts with a one-dimensional (1D) case and shows how the method is extended to a two-dimensional (2D) case. In addition, as described below, a strategy is derived by solving a regression problem in terms of the voltage of the electrode 12, in which a machine learning algorithm, specifically automatic differentiation, is used to calculate the gradient of the micro-object potential function.

[0055] The intuition behind the transition from discrete to continuous representation can be seen from the potential Specifically, V(x) can be interpreted as the discrete random function V(Y) in the discrete distribution The expected value on the quality function p i(x)It can be interpreted as the conditional probability p i(x) =Pr(Y=y i |X=x), and thus the electric potential of the micro-object can be expressed as V(x)=E[V(Y)|X=x], where V(y) is a function reflecting the electric potential at each point y. Discrete probability distribution can be regarded as the discretization of continuous probability distribution, that is where y i is the position of the ith electrode, and δ is half the electrode spacing. In terms of the error function, the parameterization of the capacitance function tells us that the conditional probability density function (pdf) is a mixture of Gaussian functions. For the spherical micro-object 11 example, the mixture has only one term, and thus the capacitance is expressed as in in Therefore, the electric potential of a micro-object in the continuous representation can be expressed as v(x) = E[V(Y) | X = x], where Y is a continuous random variable with a Gaussian distribution, i.e., Y ~ N(X,σ) and V(Y) is the function that assigns the electric potential to each point y. The potential energy can now be expressed as And thus the 1D micro-object dynamics is given by the following partial differential equation

[0056]

[0057] in and

[0058] The electrode potential modeler 25 models the function that determines the distribution of the potential as a neural network (NN) with two hidden layers of size 60 and tanh as the activation function, i.e., V(y)=V(y;β), where β is a parameter vector containing the weights and biases of the NN. The following optimization problem can be formulated:

[0059]

[0060]

[0061]

[0062] The electrode potential modeler 25 optimal solution of (1) maximizes the micro-object solution in the positive direction. For the negative direction, the sign of the cost function describing the partial derivative of the potential energy U can be changed. The electrode potential modeler utilizes the Gaussian quadrature rule commonly found in generalized chaotic polynomial theory, as described in the following literature: A. O'Hagan, "Polynomial Chaos: A Tutorial and Critique from a Statistician's Perspective", University of Sheffield, UK, May 2013; N. Wiener, "The Homogeneous Chaos", Am. J. Math., Vol. 60, No. 4, pp. 897-936, 1938; D. Xiu, D. Lucor, C.-H. Su and G. Em Karniadakis, "Performance Evaluation of Generalized Polynomial Chaos", Computational Science—ICCS 2003: International Conference, Melbourne, Australia and St. Petersburg, Russia, June 2-4, 2003, Proceedings, Part IV, Berlin, Heidelberg, 2003, pp. 346-354, the disclosures of which are incorporated by reference. The Gaussian quadrature rule provides a tool for efficiently evaluating conditional expectations. Thus, the expectation of a function of a random variable with a Gaussian distribution can be accurately approximated using the Gauss-Hermitian quadrature. Thus, the expectation of V(Y; β) is given by Given, where n is the number of sample points, y i It is the physicist's version of the Hermite polynomial H n (y) and w i Is The relevant weights given.

[0063] Similarly, the variance of V(Y; β) can be approximated as

[0064]

[0065] Therefore, the optimization problem (1) becomes

[0066]

[0067]

[0068]

[0069] The electrode potential modeler 25 performs optimization according to equation (2) using a machine learning algorithm, automatic differentiation. Specifically, the electrode potential modeler 25 can utilize the method developed by Facebook, Inc., Menlo Park, California, and described by Maclaurin et al. (“Autograd: Effortless gradients in numpy”, ICML 2015 AutoML Workshop, 2015, the disclosure of which is incorporated by reference). The automatic differentiation function of the software Autograd module is used to evaluate the cost function, which needs to be calculated as a function of NN weights and biases. , although other automatic differentiation implementations are possible. If the optimization of equation (2) is to be solved, the electrode potential modeler 25 runs an Adam gradient-based optimization algorithm that utilizes automatic differentiation results, such as described in D.P. Kingma and J. Ba, Adam: A Method for Stochastic Optimization, 2014 (although other specific implementations are possible), with a step size of α = 0.001 for 1500 iterations (although in other embodiments, other numbers of iterations and other step sizes are possible).

[0070] As described below, through experiments, the optimization problem is solved for x = 0 and the sequence of sample points n∈{10,20,30,40,50}. They all produce the same type of control strategy24, as in Ref. Fig. 9 As shown, n=30. Fig. 9 is a diagram showing the control strategy obtained by solving equation (2) according to one embodiment. The resulting control strategy is a dual-mode control strategy that exhibits a sudden change in polarity just in front of the micro-object. Due to the translational symmetry of the micro-object dynamics, it is sufficient to solve the optimization problem for only x=0.

[0071] Due to the physical limitations of the addressability of the electrode array, the electrode potential modeler 25 considers the special case of a dual-mode control strategy:

[0072]

[0073] The optimal parameters of this strategy are determined by the electrode potential modeler 25. Specifically, for b = 0.4307σ, the micro-object will move in the positive direction at the maximum speed (i.e. ), and for b = -0.4307σ, the micro-object will move in the negative direction with the maximum speed (i.e., ), where σ is the variance of the Gaussian random variable Y.

[0074] Note that it does not make sense to activate all electrodes 12, but only those in the interval [x+b, 4σ], so switching off the electrode activation after 4σ does not affect the generated force. Thus, this interval imposes an image pattern of a certain length. i ∈[x+b,4σ], the discretization control strategy is given by V i =V max Otherwise unassigned, where y i is the position of electrode i, which is exactly the control scheme used in the experiment. Therefore, the discretized control strategy 24 generated for moving the micro-object in one dimension is i ∈[x+b,4σ] is given by V i =V max Otherwise unassigned (0), where y i is the position of electrode i.

[0075] The results given above can also be extended to moving the micro-object 11 in two dimensions. The position of the micro-object 11 is given by x=(x 1 ,x 2 ) and represented by represents the position of electrode i. The dynamics of micro-object 11 becomes

[0076]

[0077] where μ is the viscosity coefficient and U(x) represents the potential energy of the micro-object.

[0078] As in the one-dimensional case, the potential energy is given by

[0079]

[0080] Given, where V(x) represents the electric potential of the micro-object, C i(x) =C(||xy (i) ||) represents the position y of the micro-object and electrode i (i) The capacitance between the i represents the potential of electrode i.

[0081] Similar to the one-dimensional case, assuming that the capacitance C i can be expressed as an unnormalized discretization of a multivariate Gaussian pdf, namely:

[0082]

[0083] in The control objective is to determine the distribution of electrode potential that moves the micro-object toward position x (d) move. Fig.10 is a diagram illustrating a control strategy 24 for moving a micro-object to a desired position in two dimensions according to one embodiment. Fig.10The scenario shown is one in which motion is parallel to y 1 axis scenario. This is sufficient because the potential distributions for other scenarios are derived by performing a rotational transformation, since the micro-object dynamics have translational and rotational symmetries. The corresponding 2D control strategy is

[0084]

[0085] All the results for the 1D case regarding the choice of b that produces the maximum force also apply to the 2D case. Note that the discretization of the ideal control strategy will introduce some errors, since the micro-object may not be exactly on top of the electrode. That is why, when the electrode is not perfectly aligned with the direction towards the desired position, the experimental strategy is based on a pattern that includes neighboring electrodes.

[0086] By using the rotational symmetry of the model, the electrode potential modeler 25 determines the 2D control strategy for any direction of motion. Assuming x (0) is the current micro-object position, and assuming x (d) is the desired micro-object position. Then, the control strategy 24 is given by:

[0087]

[0088] For positive b, and Make sure the micro-object is oriented toward the desired position x (d) motion, and for negative b, away from x (d) sports.

[0089] Thus, by taking as input the dynamic model 20, the initial micro-object position 22, and the desired micro-object position 23, the electrode potential modeler 25 can derive a control strategy for moving the micro-object 11 from the initial position 22 to the desired micro-object position 23. The control strategy 24 is designed to move a single micro-object position at a time, because the micro-objects in this design will not be close enough to each other to cause significant micro-object-micro-object interaction (the exact distance at which one micro-object 11 needs to be away from another micro-object to avoid causing an interaction that needs to be considered is determined by the geometry and potential of the micro-objects - generally, to cause a micro-object-to-micro-object interaction, the distance between the two micro-objects must be of the same order of magnitude as the distance from the micro-object 11 to the electrode 12.) The control strategy 24 generated by the electrode potential modeler is used by a projector controller 28, which is executed by one or more computing devices 16 to map the control strategy to an image 37 provided to the video projector 14, which in turn projects the image 37 to the phototransistor of the control electrode 11, thereby actuating the movement of the micro-object 11. Mapping can be performed as described in U.S. Patent Application Publication US20200207617, entitled “Micro-assembler system for manipulating micro-objects”, Plochowietz et al., published on July 2, 2020, the disclosure of which is incorporated by reference. In another embodiment, if the electrode array 35 has integrated active matrix addressing electronics, the potential on each electrode of the array is updated electronically when the control strategy 24 has been generated.

[0090] Converting the dynamic model from a discrete to a continuous formulation allows the constraints of the model to be rewritten as expectations. This, in turn, allows the model to be rearranged in a more efficient way, allowing the optimal electrode potentials to be derived more efficiently using automatic differentiation. Fig.11 is a flow chart showing a method 50 for multi-object micro-assembly control by means of a digital computer. The method 50 may be used Figure 1 The system is executed. Parameters of the system are obtained, such as the diameter of the micro-object, the size of the electrode, the dielectric fluid constant, the initial position and material of the one or more micro-objects and electrodes to be moved (which can be obtained using a high-speed camera or another sensor), and the height of the micro-object, although other parameters are possible (step 51). A dynamic model of the capacitance-based motion of the micro-object is created using the parameters (step 52), as described below with reference to Fig.10 Further described and referenced above Figure 1 Obtain the desired position of at least one of the micro-objects (step 53). Figure 1Described, use the initial position of one of the micro-object and the desired position of this micro-object to determine the control strategy (step 54).In short, the dynamic model with distributed parameters is converted to the continuous representation of the dynamic model, and automatic differentiation is used for solving the optimization problem together with the Gauss quadrature rule, to derive the electrode potential required for moving the micro-object to the desired position in one or two dimensions.The electrode potential derived is mapped to the image for video projector display (step 54), and the electrode in the control electrode array is to cause the micro-object to move to the desired position (step 55).If other movement (step 57) is needed, such as if other micro-objects need to be moved, or if the micro-object moved does not reach the desired position (determined using a high-speed camera or another sensor), then the method turns back to step 54.If other movement (step 57) is not needed, then method 50 ends.

[0091] In further embodiments, if the electrode array 35 has integrated active matrix addressing electronics, the potential on each electrode of the array is updated electronically when the control strategy 24 has been generated by the image projected by the projector in step 54 .

[0092] Since the systems and methods use only one micro-object at a time, and the micro-objects are not close enough to produce micro-object-micro-object interactions, the dynamic model needs to consider only the interaction between one micro-object and the electrode. The capacitance-based motion model of the micro-object considers both the capacitance between the micro-object and the electrode and the capacitance between two different micro-objects. Fig.12 is a diagram showing a method for using a Fig.11 A flow chart of a routine 60 for modeling micro-object motion using capacitance in method 50. Figure 1 As described above, the capacitance between the electrode and the micro-object is simulated, such as a COMSOL simulation (step 61). Figure 1 As described above, a function describing the capacitance correlation between the electrode and the micro-object is defined based on the simulation of the capacitance between the micro-object and the electrode (step 62), thereby ending the routine 60.

[0093] The systems and methods described above utilize a dual-mode control strategy derived using a machine learning algorithm and Gaussian quadrature rule to derive the necessary electrode potentials. Additional mathematical reasoning and experimental details behind the derivation and application of the control strategy are given below. The reasoning and details are provided for the purpose of illustration and not limitation.

[0094] Control Design: 1D Case

[0095] As mentioned above, the intuition behind the transition from discrete to continuous representation can be found in the micro-object potential. Specifically, V(x) can be interpreted as the discrete random function V(Y) in the discrete distribution The expected value on the quality function p i(x) It can be interpreted as the conditional probability p i(x) =Pr(Y=y i |X=x), and thus the electric potential of the micro-object can be expressed as V(x)=E[V(Y)|X=x], where V(y) is a function reflecting the electric potential at each point y. Discrete probability distribution can be regarded as the discretization of continuous probability distribution, that is where y i is the position of the ith electrode, and δ is half the electrode spacing. The parameterization of the capacitance function in terms of the error function tells us that the conditional probability density function (pdf) is a mixture of Gaussian functions. For the spherical example, the mixture has only one term, and so the capacitance is expressed as in in Therefore, the electric potential of a micro-object in the continuous representation can be expressed as where Y is a continuous random variable with a Gaussian distribution, i.e. Y~N(X,σ) and V(Y) is the function that assigns the electric potential to each point y. The potential energy can now be expressed as And the 1D micro-object dynamics is given by the following partial differential equation

[0096]

[0097] in and

[0098] The following results show that the dual-mode potential distribution is sufficient to move the micro-object to the desired position.

[0099] Proposition 1 assumes that the function describing the potential distribution is chosen as

[0100]

[0101] Where V max Then, for b = 0.8769σ, the micro-object will move in the positive direction (i.e., x>0) with the maximum speed, and for b = -0.8769σ, the micro-object will move in the negative direction (i.e., x<0) with the maximum speed, where σ is the variance of the Gaussian random variable Y.

[0102] Check. Fig.13 See the graph of the function V(y) defined by equation (4) Fig.13 is a diagram showing a dual-mode control strategy according to one embodiment, where x is the micro-object position and the voltage polarity changes at x+b. The Gaussian distribution of Y enables calculation of and a closed form expression for the potential energy.

[0103] Specifically:

[0104]

[0105] To evaluate the partial derivatives of the potential, we evaluate the energy function at the points x and x+ε and then compute the limit therefore,

[0106]

[0107] The perturbed average voltage is given by:

[0108]

[0109] This allows calculation of the potential energy at x+ε, which is

[0110]

[0111] at last,

[0112]

[0113] Expression (5) shows is an effective control strategy, resulting in the same energy function and induced force. Therefore, the maximum speed is given by The maximum value of Fig.14 The force dependence on b is plotted. Fig.14 is shown for V max = 1 and σ = 0.1, the force distribution as a function of b. The force derivative with b as the two critical points is the equation The solution is calculated numerically. Specifically, a series of values ​​of σ are generated and the critical points corresponding to them are calculated. Thus, the critical point linearly related to σ is found, namely b * =±0.8769σ.

[0114] The previous results show that the dual-mode voltage distribution on both sides of the electrode 11 does ensure that the micro-object moves in the desired direction. More importantly, and not surprisingly, the results show that only a limited number of electrodes are relevant, namely the electrodes whose positions are in the interval [-4σ, 4σ]. The control strategy in the continuous representation of the electrodes is actually implemented by discretizing the control strategy, resulting in

[0115]

[0116] where y i is the position of electrode i, and x is the position of the micro-object.

[0117] Control Policy

[0118] As described above, a control strategy based on the experimental design of projecting an image with a specific pattern is used to push a micro-object in a desired direction. The following description analytically shows why the control strategy actually works and formally rediscovers the shape of the pattern. As in the previous section, the 1D case is considered and then extended to the 2D case. Consider the control strategy given by:

[0119]

[0120] Reference Fig.15 Graphically depict the control strategy. Fig.15 is a diagram showing a dual-mode control strategy according to an embodiment, where x is the position of the micro-object and the electric potential starts to become non-zero at x + b.

[0121] In the control strategy of the experimental design, for all points y ≥ x + b, the voltage does not remain constant, and for y < x + b, the electrode potential is freely floating. It is assumed here that the frequency of image projection on the array is small enough so that the capacitors associated with the electrode photodiodes are fully discharged when they are not activated (i.e., the projection on them is black). The following results summarize the effect of this new strategy on the movement of the micro-object.

[0122] Proposition 2. Assume that the function describing the electric potential distribution is chosen as (6), where V max the maximum absolute value of. Then, for b = 0.4307σ, the micro-object will move in the positive direction with the maximum speed (i.e., ), and for b = -0.4307σ, the micro-object will move in the negative direction with the maximum speed (i.e., ), where σ is the variance of the Gaussian random variable Y.

[0123] Test. This test follows the same steps as the test of Proposition 1. The main difference is that now the integration is only performed within the interval [x + b, ∞). As mentioned before, the Gaussian distribution of Y allows the calculation of a closed-form expression for the potential energy. Specifically, the potential energies at U(x) and U(x + ε) are given by the following respectively

[0124] -

[0125] and

[0126]

[0127] Taking ε to zero, we obtain the force expression:

[0128]

[0129] refer to Fig.16 The force is shown as a function of the variable b. Fig.16 is a graph showing the force distribution as a function of b for V_max=1 and σ=0.1 using the control strategy of equation (6). Fig.16 Two critical points are also depicted, which maximize the velocity in the desired direction. There are two important differences compared to the strategy introduced in equation (4). The critical point is given by b * = ±0.4307σ, where these points are numerical estimates, and the force magnitude is significantly reduced (by more than 80%).

[0130] Note that it does not make sense to activate all electrodes, but only those in the interval [x+b,4σ], so cutting off electrode activation after 4σ will not affect the generated force. i ∈[x+b,4σ], the discretization control strategy is given by V i =V max Otherwise unassigned, where y i is the position of electrode i, which is exactly the control scheme used in the experiment. The control strategy of the experimental design was implemented by considering some physical limitations of the actuation mechanism, which do not allow setting both positive and negative polarity for the electrode potential.

[0131] Optimal Control Design Enabled by Machine Learning

[0132] The above provides a formal test that the dual-mode control strategy is sufficient to induce micro-object motion in the desired direction. The interesting question is whether such a strategy is optimal. Although the following description does not analytically prove optimality, it provides a method for solving the regression problem of the potential distribution of the array using a deep learning optimization framework (as described below: Dougal Maclaurin, David Duvenaud, and Ryan P Adams, "Autograd: Effortless gradients in numpy.", In ICML 2015 AutoML Workshop, 2015; and Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer., "Automatic differentiation in pytorch.", 2017, the disclosures of which are incorporated by reference) to solve the regression problem. As seen below, the optimization results strengthen the conjecture that the control strategy derived by equation (4) is indeed the best.

[0133] The function determining the distribution of the potential is modeled as a neural network (NN) with two hidden layers of size 60 and tanh as activation function, i.e., V(y) = V(y; β), where β is a parameter vector containing the weights and biases of the NN. The following optimization problem previously provided in equation (1) is formulated.

[0134]

[0135]

[0136]

[0137] The optimal solution of (1) maximizes the micro-object solution in the positive direction. For the negative direction, the sign of the cost function describing the partial derivative of the potential energy U can change. Note that, unlike the case of the strategy using equation (4), the conditional expectation can no longer be evaluated analytically. To address this challenge, Gaussian quadrature rules are used, which, as mentioned above, often appear in the generalized polynomial theory of chaos as described by: Anthony O'Hagan, Polynomial chaos: A tutorial and critique from a statistician's perspective, Technical report, University of Sheffield, UK, May 2013; Ralph C. Smith, Uncertainty Quantification: Theory, Implementation, and Applications; Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 2013; Norbert Wiener, The Homogeneous Chaos, American Journal of Mathematics, Vol. 60, No. 4, pp. 897-936, 1938; Dongbin Xiu, Didier Lucor, C.-H. Su, and George Em Karniadakis, Performance evaluation of generalized polynomial chaos, Computational Mathematics, vol. 60, no. 4, pp. 897-936, 1938. Science—ICCS 2003: International Conference, Melbourne, Australia and St. Petersburg, Russia, June 2-4, 2003, Proceedings, Part IV, pp. 346-354, Berlin, Heidelberg, 2003, Springer Berlin Heidelberg, the disclosures of which are incorporated by reference. The Gaussian quadrature rule provides a tool for efficiently evaluating conditional expectations. Thus, the expectation of a function of a random variable with a Gaussian distribution can be accurately approximated using the Gauss-Hermite quadrature. Thus, the expectation of V(Y; β) is given by

[0138]

[0139] Given, where n is the number of sample points, y iIt is the physicist's version of the Hermite polynomial H n(y) The root of i Is The relevant weights given.

[0140] Similarly, the variance of V(Y; β) can be approximated as

[0141]

[0142] Therefore, the optimization problem (1) becomes as given previously in equation (2):

[0143]

[0144]

[0145]

[0146] Developed by Facebook, Menlo Park, CA (and described by Maclaurin et al., "Autograd: Effortless gradients in numpy", ICML 2015 AutoML workshop, 2015) The automatic differentiation function of the Autograd module of the software is used to evaluate the cost function, which requires calculating the cost function as a function of the NN weights and biases. To solve Eq. 2, the Adam gradient-based optimization algorithm is run for 1500 iterations with a step size of α = 0.001. The optimization problem is solved for a sequence of x = 0 and the number of sample points n∈{10,20,30,40,50}. They all produce the same type of control strategy, as in Ref. Fig. 9 As shown, n = 30. We recover the dual-mode control strategy introduced in equation (4), which shows a sudden change of polarity just in front of the micro-object. Due to the translational symmetry of the micro-object dynamics, it is sufficient to solve the optimization problem only for x = 0.

[0147] Extension to 2D control strategies

[0148] In this section, we show how the previous results can be extended to the 2D case. 1 ,x 2 ) represents the position of the micro-object, and is represented by represents the position of electrode i. The dynamics of the micro-object becomes

[0149]

[0150] where μ is the viscosity coefficient and U(x) represents the potential energy of the micro-object. As a 1D case, the potential energy is given by

[0151]

[0152] Where V(x) represents the electric potential of the micro-object, C i(x) =C(||xy (i) ||) represents the position y of the micro-object and electrode i (i) The capacitance between the i represents the potential of electrode i. Similar to the 1D case, assuming that the capacitance C i can be expressed as an unnormalized discretization of a multivariate Gaussian pdf, namely:

[0153]

[0154] in The control objective is to determine the distribution of electrode potential that moves the micro-object toward position x (d) Move. Reference Fig.10 The scenario shown here is one in which the motion is parallel to y 1 axis. This is sufficient because the potential distributions for other scenarios are derived by performing a rotational transformation, since the micro-object dynamics have translational and rotational symmetries.

[0155] We use the following two-mode control strategy:

[0156]

[0157] The electric potential of the micro-object is calculated as the average value of the electric potential function V(y), that is,

[0158]

[0159] The potential energy can be calculated in closed form and is given by

[0160]

[0161] To evaluate the gradient of the potential, calculate the partial derivative of U(x). Thus,

[0162]

[0163] in

[0164]

[0165] therefore,

[0166]

[0167] Similarly, U(x 1 ,x 2 ) is the partial derivative with respect to x 2The result of the calculation is zero, that is

[0168]

[0169] Note that the control strategy is only 1 The forces are generated on the axis and the control strategy has the same exact expression as for the forces generated in the 1D case. All the results for the 1D case regarding the choice of b that produces the maximum force also apply to the 2D case. Note that the discretization of the ideal control strategy will introduce some errors, since the micro-object may not be exactly on top of the electrode. That is why, when the electrode is not perfectly aligned with the direction towards the desired position, the experimental strategy is based on a pattern that includes neighboring electrodes.

[0170] The general 2D control strategy can be summarized in the following results.

[0171] Proposition 3: Assume x (0) is the current micro-object position, and assuming x (d) is the desired micro-object position. Then the control strategy

[0172]

[0173] For positive b, and Make sure the micro-object is oriented toward the desired position x (d) motion, and for negative b, away from x (d) sports.

[0174] Test. This is done by applying a rotation transformation so that and Can make and apply strategy (11). We also note that In the connection x (0) and x (d) The straight lines are perpendicular to each other and pass through the same straight line at a distance x. (0) Point x at distance b (b) .

[0175] While the invention has been particularly shown and described with reference to embodiments thereof, it will be understood by those skilled in the art that the foregoing and other changes in form and details may be made therein without departing from the spirit and scope of the invention.

Claims

1. A system for micro-assembly control enabled by machine learning with a digital computer, the system comprising: One or more processors implementing one or more machine learning algorithms, the one or more processors configured to: obtaining one or more parameters of a system for positioning a micro-object, the system comprising a plurality of electrodes, each of the electrodes being programmable, the electrodes being configured to cause movement of the micro-object when one or more electrical potentials are generated by one or more of the electrodes when the micro-object is in proximity to the electrodes; modeling the capacitance between the micro-object and the electrode using the parameters; estimating the position of the micro-object using a sensor; receiving another position of the micro-object; defining a continuous representation of a dynamic model of movement of the micro-object due to an electrode potential generated by at least one of the plurality of electrodes using the capacitance, wherein the dynamic model is associated with a constraint represented as a desired constraint; applying one of the machine learning algorithms to at least a portion of the continuous representation of the dynamic model to perform an optimization of an electrode potential generated by at least one electrode of the plurality of electrodes to move the micro-object from the estimated position to the additional position; as well as At least one electrode of the plurality of electrodes is actuated using the optimized electrode potential to cause the movement of the micro-object from the estimated position to the further position. 2 . The system of claim 1 , wherein one of the constraints comprises a potential energy of the micro-object, and the potential energy is approximated using a Gaussian quadrature rule during the optimization.

3. The system of claim 2, wherein the one or more processors are further configured to: The function determining the optimized distribution of electrode potentials is represented as a neural network associated with weights and biases.

4. The system of claim 3, wherein the one machine learning algorithm is automatic differentiation, and the one or more processors are further configured to: Partial derivatives of the potential as a function of the weights and the bias associated with the neural network are calculated using the automatic differentiation.

5. The system of claim 3, wherein the neural network comprises two hidden layers and uses tanh as an activation function.

6. The system of claim 3, wherein the additional position is spaced apart from the estimated position by a distance in a direction, the one or more processors being further configured to: receiving an additional location spaced from the estimated location by the distance in a direction opposite to the direction in which the additional location is spaced from the estimated location; and The sign of the function is changed to optimize the potential for moving the micro-object from the estimated position to the further position.

7. The system of claim 1 , wherein the movement from the estimated position to the additional position is in one dimension, the one or more processors being further configured to: receiving an additional position spaced apart from the estimated position in two dimensions; and A rotational transformation is applied to the optimized electrode potential to derive the electrode potential generated by the electrode to move the micro-object from the estimated position to the further position.

8. The system of claim 1, wherein the one or more processors are further configured to: performing a plurality of simulations of the capacitance between the electrode and the micro-object; and A function is defined which describes the capacitance between the micro-object and each of the electrodes as a function of the distance between the micro-object and the electrode.

9. The system of claim 1, one or more of the processors comprising one or more of at least one central processing unit (CPU) and at least one graphics processing unit (GPU).

10. The system of claim 1, wherein at least one electrode of the plurality of electrodes is rectangular.

11. A method of micro-assembly control enabled by machine learning with a digital computer, the method comprising: obtaining, by one or more processors implementing one or more machine learning algorithms, one or more parameters of a system for locating a micro-object, the system comprising a plurality of programmable electrodes configured to cause movement of the micro-object when one or more electrical potentials are generated by one or more of the electrodes when the micro-object is in proximity to the electrodes; modeling, by one or more of the processors, a capacitance between the micro-object and the electrode using the parameters; estimating, by one or more of the processors, a position of the micro-object based on at least one sensor measurement; receiving, by one or more of the processors, further positions of the micro-objects; defining, by one or more of the processors, a continuous representation of a dynamic model of movement of the micro-object due to an electrode potential generated by at least one of the plurality of programmable electrodes using the capacitance, wherein the dynamic model is associated with a constraint represented as a desired constraint; applying, by one or more of the processors, one of the machine learning algorithms to at least a portion of the continuous representation of the dynamic model to perform optimization of an electrode potential generated by at least one of the plurality of programmable electrodes to cause the micro-object to move from the estimated position to the additional position; as well as At least one programmable electrode of the plurality of programmable electrodes is actuated using the optimized electrode potential to cause the movement of the micro-object from the estimated position to the further position.

12. The method of claim 11, wherein one of the constraints comprises a potential energy of the micro-object, and the potential energy is approximated using a Gaussian quadrature rule during the optimization.

13. The method according to claim 12, further comprising: The function determining the optimized distribution of electrode potentials is represented as a neural network associated with weights and biases.

14. The method of claim 13, wherein the one machine learning algorithm is automatic differentiation, the method further comprising: Partial derivatives of the potential as a function of the weights and the bias associated with the neural network are calculated using the automatic differentiation.

15. The method of claim 13, wherein the neural network comprises two hidden layers and uses tanh as an activation function.

16. The method of claim 13, wherein the additional position is spaced a distance in a direction from the estimated position, the method further comprising: receiving an additional location spaced from the estimated location by the distance in a direction opposite to the direction in which the additional location is spaced from the estimated location; as well as The sign of the function is changed to optimize the potential for moving the micro-object from the estimated position to the further position.

17. The method of claim 11, wherein the movement from the estimated position to the further position is in one dimension, the one or more processors being further configured to: receiving an additional position spaced apart from the estimated position in two dimensions; and A rotational transformation is applied to the optimized electrode potential to derive the electrode potential generated by the electrode to move the micro-object from the estimated position to the further position.

18. The method according to claim 11, further comprising: performing a plurality of simulations of the capacitance between the electrode and the micro-object; as well as A function is defined which describes the capacitance between the micro-object and each of the electrodes as a function of the distance between the micro-object and the electrode.

19. The method of claim 11, the one or more of the processors comprising one or more of at least one central processing unit (CPU) and at least one graphics processing unit (GPU).

20. The method of claim 11, wherein at least one programmable electrode of the plurality of programmable electrodes is rectangular.

Images