System and method for micro-object density distribution control using machine learning performed by digital computers

Abstract

The present invention is entitled "System and method for micro-object density distribution control implemented by machine learning with the aid of a digital computer". The present invention provides a system and method for shaping the micro-object density distribution (the density of assembling micro-objects in a specific spatial region). A high-speed camera tracks the existing object density distribution. An array of electrodes controlled by phototransistors is used to generate a dynamic potential energy map for manipulating objects through both DEP forces and EP forces, and a video projector is used to actuate the array. One or more computing devices are used to: process images captured by the camera to estimate the existing density distribution of the object; receive the desired density distribution of the micro-objects; define a model that describes the change in the density of the micro-objects over time due to capacitance-based interactions; generate a sequence of electrode potentials that, when generated, will minimize the error between the existing density distribution and the desired density distribution; and use the sequence of electrode potentials to actuate the electrodes.

Description

Technical Field

[0001] The present application relates generally to micro-assembly control, and in particular, to systems and methods for micro-object density distribution control implemented by machine learning with the aid of digital computers. Background Art

[0002] Micron- and nanometer-scale particle manipulation has received much research interest. The degree to which the assembly of micro-objects (i.e., objects whose size is measured in microns) and nano-objects (i.e., objects whose size is measured in nanometers) can be controlled can produce significant differences in many technologies (including in micro-manufacturing, biology, and medicine). For example, the manufacture of reconfigurable circuits can be improved by being able to accurately control the position of micro-objects (such as capacitors and resistors) to manufacture circuits with desired behaviors. Similarly, the production of photovoltaic solar cell arrays can benefit from being able to put photovoltaic cells with certain qualities into specific positions on the array. Such batteries may be too small to allow the desired placement of batteries via human or robot manipulation, and another type of transport mechanism is required. The micro- and nano-assembly of particles can also be used for the microstructure of engineered materials (such as biological cells assembled into tissues). There are many other technical fields in which increasing the control of micro-nano object assembly can provide significant benefits.

[0003] A particular challenge associated with microassembly is to generate a specific desired density of micro-objects in a specific spatial region; once achieved, this density distribution can be used as the basis for individual micro-object manipulation that can achieve the desired micro-object manipulation. However, simultaneous control and tracking of individual micro-objects is computationally expensive, and existing technologies do not allow for control of the density distribution of micro-objects with the degree of control and computational efficiency desired for many practical applications.

[0004] For example, uncontrolled mechanical agitation is often used for directed particle assembly. However, this technique cannot achieve the near 100% yield necessary for certain industrial applications, such as electronics assembly.

[0005] Previous studies have also attempted to use electric fields to directly move micro-objects. For example, Matei et al., "Micro-scale chiplets position control", Journal of Microelectromechanical Systems, 28(4): 643-655, August 2019, and Matei et al., "Towards printing as an electronics manufacturing method: Micro-scale chiplet position control", 2017 American Control Conference (ACC), pp. 1549-1555, May 2017, disclose micro-chip control strategies based on a one-step model predictive control method. The disclosed one-dimensional model is based on capacitance, but the actuation mechanism used is based on spiral electrodes, which limits the number of simultaneously actuated electrodes that can be used by the model. In addition, it is assumed that the size of the particle whose movement is controlled is small enough so as not to interfere with the electric field, which can limit the applicability of the described technology to larger particles.

[0006] Likewise, Wang et al., "Dielectrophoretic manipulation of cells with spiral electrodes," Biophysical Journal, 72(4): 1887-1899, 199, describes a study of the effects of dielectrophoresis on cancer cells. This study assumes that the particles involved are spherical and small enough that the electric field is not affected by their presence, limiting the applicability of the described technique.

[0007] Tara D. Edwards and Michael A. Bevan, "Controlling colloidal particles with electric fields," Langmuir, 30(36): 10793-10803, 2014. PMID: 24601635 ("Edwards"), whose disclosure is incorporated by reference, describes control schemes for individual and integrated control of colloids. Specifically, Edwards shows how inhomogeneous electric fields can be used to manipulate individual colloidal particles (1 μm to 3 μm in diameter) and collections thereof in water and sodium hydroxide solutions by electrophoresis and electroosmosis. The relative size of the colloids with respect to the electrodes used to generate the fields, the medium in which the particles are immersed, and the resulting mathematical models do not allow the use of the described techniques in certain industrial applications. In particular, the described techniques are not suitable for assembling micro-objects, even slightly larger than those discussed in the Edwards paper. In addition, the control scheme used involves high-frequency signals (MHz), which further limits the applicability of such techniques.

[0008] Similarly, Qian et al., "On-demand and location selective particle assembly via electrophoretic deposition for fabricating structures with particle-to-particle precision," Langmuir, 31(12): 3563-3568, 2015. PMID: 25314133, the disclosure of which is incorporated by reference, demonstrates single particle precision and location selective particle deposition, where electrophoretic forces are the primary drive for particle (2 μm polystyrene beads) manipulation. The control scheme employed is based on building a large energy well close to the desired location of the nanoparticle. However, the described techniques do not allow for proper sorting and placement of individual objects. Furthermore, the described techniques do not allow for adequate manipulation of asymmetric objects (such as semiconductor chips) that require orientation control for building electronic systems.

[0009] Several studies such as Xue et al., "Optimal design of a colloidal self-assembly process," IEEE Transactions on Control Systems Technology, 22(5): 1956-1963, September 2014, and Xue et al., "Mdp based optimal control for a colloidal self-assembly system," 2013 American Control Conference (ACC), pp. 3397-3402, June 2013, the disclosures of which are incorporated by reference, describe controlling a random colloidal assembly process that drives the system to a desired high crystallinity state and is based on a Markov decision process optimal control strategy. The dynamic model is based on the Langevin equation parameterized by the actuator. However, in these studies, individual particles are not manipulated directly, and this approach can be used when the assembly circuit is unclear. In addition, the size of the particles used (about 3 μm in diameter) is small, to the extent that they have little interference with the electric field fully shaped by the actuation potential. Furthermore, the time scale for achieving the desired state will make the goal of achieving high throughput using this approach challenging to achieve.

[0010] Other self-assembly control methods, such as those described by Grzelczak et al., “Directed self-assembly of nanoparticles,” ACS Nano, 4(7): 3591-3605, 2010. PMID: 20568710; Paulson et al., “Control of self-assembly in micro- and nano-scale systems,” Journal of Control Processes, 27: 38-49, 2015; Mastrangeli et al., “Automated real-time control of fluidic self-assembly of microparticles,” 2014 IEEE International Conference on Computer Aided Design, pp. 5860-5865, May 2014; and Paulson et al., “Control of self-assembly in micro- and nano-scale systems,” Journal of Control Processes, 27: 38-49, 2015. systems), Journal of Control Processes, 27: 38-49, 2015, the disclosure of which is incorporated by reference.

[0011] A water-based solution in which the particles are immersed is a common choice of control medium, such as described by Edman et al., "Electric field directed assembly of indium gallium arsenide led onto silicon circuitry," IEEE Photonics Technology Letters, 12(9): 1198-1200, September 2000, and Tolley et al., "Dynamically programmable fluidic assembly," Applied Physics Letters, 93(25), 2008, the disclosures of which are incorporated by reference. In such cases, both electrophoretic forces and fluid motion by electroosmotic flow are used to drive the particles. However, water does not behave like a dielectric and therefore cannot generate an electric field, resulting in lower forces for moving objects, significantly limiting the size of objects that can be moved using this setup.

[0012] Accurate control of cells, quantum dots, and nanowires based on electroosmosis was used in Mathai et al., “Simultaneous positioning and orientation of single nano-wires using flow control,” RSC Adv., 3:2677-2682, 2013, and Probst et al., “Flow control of small objects on chip: Manipulating live cells, quantum dots, and nanowires,” IEEE Control Systems, 32(2):26-53, April 2012, the disclosures of which are incorporated by reference. The authors used a linear model of the electrode potential, and the effect of the particles on the electric field distribution was negligible. However, when the driving force is primarily dielectrophoresis, the linearity exhibited in the electrode potential does not hold, limiting the applicability of these techniques. Further, the linearity exhibited may not remain constant with respect to micrometer-scale objects.

[0013] Finally, Zemánek et al., "Feedback-controlled dielectrophoretic micromanipulation," 2018 International Conference on Manipulation, Automation, and Robotics at Small Scale (MARSS), July 2018, whose disclosure is incorporated by reference, describes dielectrophoresis-based feedback for microsphere manipulation. The authors used a simulated annealing approach to solve the optimal control problem, where they took advantage of the spherical shape of the particles when building a model of the system. However, the applicability of this technique is limited by its reliance on the spherical shape of the particles being moved.

[0014] Therefore, there is a need for an easily scalable, computationally efficient way to control the density distribution of micro-objects. Summary of the invention

[0015] The present invention provides systems and methods that allow shaping of micro-object density distribution (how densely micro-objects are assembled in a specific spatial region). A high-speed camera tracks the existing object density distribution. An array of phototransistor-controlled electrodes is used to generate a dynamic potential energy map for manipulating objects through both DEP forces and EP forces, and a video projector is used to actuate the array. One or more computing devices are used to: process images captured by the camera to estimate the existing density distribution of the object; receive the desired density distribution of the micro-object; define a model that describes the change in the density of the micro-object over time due to capacitance-based interactions; generate a sequence of electrode potentials that will minimize the error between the existing density distribution and the desired density distribution when generated; and map the sequence of electrode potentials to an image that is projected on the array using a video projector. The projected image activates or deactivates the electrode, as indicated by the electrode potential. Machine learning algorithms (such as automatic differentiation) are used to increase the efficiency of optimizing the sequence of electrode potentials.

[0016] In one embodiment, a system and method for micro-object density distribution control by means of a digital computer is provided. One or more parameters of a system for locating a plurality of micro-objects are obtained by one or more processors configured to execute computer executable code, the system comprising a plurality of programmable electrodes, the electrode being configured to induce movement of the micro-object when the micro-object approaches the electrode after one or more potentials are generated by one or more electrodes in the electrode. A model describing the change of the density of the micro-object over time due to capacitance-based interactions is defined by one or more processors in the processor based on the parameters of the system. The density distribution of the micro-object is estimated by one or more processors in the processor using at least one sensor. The target density distribution of the micro-object is received by one or more processors in the processor. Model predictive control (MPC) optimization is performed by one or more processors in the processor to derive a sequence of the electrode potentials based on the density model, the sequence of the electrode potentials will be generated by at least some of the electrodes for moving at least some of the micro-objects in the micro-object, thereby minimizing the error between the estimated density distribution and the target density distribution. At least some of the electrodes are actuated by one or more processors in the processor to generate the sequence of the electrode potentials.

[0017] Other embodiments of the present invention will become apparent to those skilled in the art from the following detailed description, wherein embodiments of the present invention are described by illustrating the best mode contemplated for carrying out the present invention. As will be appreciated, the present invention is capable of other and different embodiments, and its several details can be modified in various obvious aspects, all without departing from the spirit and scope of the present invention. Therefore, the drawings and detailed description should be regarded as illustrative rather than restrictive in nature. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a diagram illustrating a control objective where initially the micro-objects have a uniform distribution and then converge to a multivariate Gaussian distribution centered at zero and with a given covariance matrix.

[0019] Figure 2 is a block diagram showing a system for micro-object density distribution control implemented by machine learning with the aid of a digital computer.

[0020] Figure 3 is a diagram showing a video projector and a Figure 2 Diagram of the interaction between the phototransistor array.

[0021] Figure 4 is a diagram showing a discrete grid superimposed on a continuous potential function, where the center of each grid represents an electrode.

[0022] Figure 5 It is shown that according to one embodiment Figure 2 A diagram of the system's capacitance-based interactions between a small chip and five electrodes.

[0023] Figure 6 is a diagram showing a COMSOL model of two conductors (spherical chiplet and electrode) for capacitance calculation according to one embodiment.

[0024] Figure 7 A capacitance function fitted by training data generated by a COMSOL simulation is shown according to one embodiment.

[0025] Figure 8 is a graph showing a 2D capacitance function determined from a 1D model using symmetry according to one embodiment.

[0026] Fig. 9 is a flow chart showing a method for micro-object density distribution control achieved by machine learning with the aid of a digital computer.

[0027] Fig.10 is a diagram showing a method for setting a Fig. 9 A flow chart of a routine used in the method to manage a dynamic model of chiplet density changes over time.

[0028] FIG. 11A to FIG. 11F is a graph showing the chiplet density distribution of the first example at different times.

[0029] FIG. 12A to FIG. 12F Shows the method used to create FIG. 11A to FIG. 11FEvolution of the potential of the electrodes with a small chip density distribution over time.

[0030] FIG. 13A to FIG. 13F is shown for creating FIG. 11A to FIG. 11F Plot of the force vector field over time for the chiplet density distribution.

[0031] FIG. 14A to FIG. 14F is a graph showing the chiplet density distribution of the second example at different times.

[0032] FIG. 15A to FIG. 15F Shows the method used to create FIG. 14A to FIG. 14F Evolution of the potential of the electrodes with a small chip density distribution over time.

[0033] FIG. 16A to FIG. 16F is shown for creating FIG. 14A to FIG. 14F Plot of the force vector field over time for the chiplet density distribution. DETAILED DESCRIPTION

[0034] The simultaneous control and tracking of individual chiplets is computationally expensive. The computational cost of moving multiple micro-objects simultaneously can be reduced by first performing micro-object control at the macro level: tracking and controlling the micro-object density, and changing the electrode potential to make the density reach the desired shape. Figure 1 Examples of control objectives are graphically depicted. Figure 1 is a diagram illustrating a control objective, where initially the micro-objects have a uniform distribution and subsequently converge to a multivariate Gaussian distribution, centered at zero and with a given covariance matrix. Such control objectives can be achieved using the system and method described below, which utilizes an array of electrodes that generate an electric field that causes the micro-objects to move. In order to design the sequence of electrode potentials generated by the electrodes required to achieve the desired density distribution, a macro-level mathematical model is first generated based on the physical laws that govern the behavior of micro-objects under electric fields. This macro-model describes the evolution of the density of micro-objects due to changing the electric field through the electrode potentials, f(x, t). The goal of density control is to shape the density of micro-objects into a desired pattern, f d (x, t) (e.g., Gaussian). Model Predictive Control (MPC) problem, where the loss function is the Kullback-Leibler divergence between the current micro-object density and the desired micro-object density over some time horizon.

[0035] Figure 2is a block diagram showing a system 10 for micro-object density distribution control implemented by machine learning with the aid of a digital computer. System 10 allows for coordinated assembly involving multiple objects 11 (such as micro-objects). The size of the objects 11 varies between the nanoscale (<1 μm) and the microscale (up to hundreds of μm), but other sizes are possible. The objects 11 can be spherical, but other shapes are also possible, such as rectangular shapes, but other shapes are also possible. In one embodiment, the diameter of the spherical object 11 may be 20 μm to 50 μm, but other diameters are also possible. In one embodiment, the size of the rectangular object 11 may be 200 μm×300 μm×100 μm, but other sizes are also possible. The object 11 is immersed in a dielectric fluid (not shown) contained in a housing (not shown), wherein the membrane layer 38 is lower than the dielectric fluid contained in the housing. In one embodiment, the dielectric fluid (not shown) is manufactured by ExxonMobil Chemical Company of Spring, Texas. M, but other dielectric fluids are also possible. The dielectric fluid may include one or more additives, such as di-2-ethylhexyl sulfosuccinate (AOT) charge director molecules, but other additives are also possible. Object 11 can be made of aluminum oxide (AlOx), but other materials are possible. Each object in object 11 can be a semiconductor chip, an integrated circuit, a particle, a nano device or structure, or any object that the array 35 of electrodes 12 can physically manipulate. In the description below, object 11 is interchangeably referred to as a chiplet 11 or a micro-object 11, but other ways of referring to object 11 are possible. In one embodiment, film 38 can be a 50 μm thick perfluoroalkoxy (PFA) film, but other types of films of other thicknesses are possible.

[0036] Below the suspended chiplet 11 are a plurality of electrodes 12 forming an array 35, the electrodes being configured to generate a dynamic potential energy map for manipulating objects via dielectrophoretic ("DEP") forces and electrophoretic ("EP") forces. A membrane is laminated over the electrodes 12. In one embodiment, the electrodes may be square in shape and made of copper, but other shapes and materials are possible. In one embodiment, the square electrodes 12 may be sized to be 16 μm in width and 100 nm in thickness, but in other embodiments, other sizes are possible. The array 35 may include multiple rows of electrodes 12, with each row including multiple electrodes 12.

[0037] The potential generated by the electrode 12 is controlled by an array 13 of phototransistors, which includes an active matrix phototransistor backplane disposed on glass. A plurality of phototransistors on the backplane form an array 13, wherein each of the phototransistors in the array 13 controls the potential generated by a single electrode 12. Specifically, each of the phototransistors in the array 13 is attached to one electrode 12. The array 13 of phototransistors may have additional features, such as those described in Rupp et al., "Chiplet micro-assembly printer", 2019 IEEE 69th Electronic Components and Technology Conference (ECTC), pp. 1312-1315, May 2019, the disclosure of which is incorporated by reference.

[0038] The array 13 is optically addressed by a video projector 14 to enable dynamic control of the electrostatic potential and manipulation of the positions of multiple chiplets 11 simultaneously. Figure 3 As shown, a video projector 14 is used to address each phototransistor controlled electrode 12, thereby allowing easy scaling factor changes and stitching into larger arrays. Figure 3 FIG. 1 is a diagram showing a video projector 14 and a Figure 2 FIG. 1 is a diagram of the interaction between the phototransistor array 13 of FIG. 1. The video projector 14 actuates the electrodes 12 by projecting a predefined pattern (an image 37 generated based on a control input): an electric potential will be generated by the electrodes to achieve the desired movement of the chiplet 11, as further described below. The pixelated light constituting the image projected by the projector 14 charges each individual phototransistor 13 in the array to the necessary extent so that the electrode 12 charged by that phototransistor produces the desired electric potential. Additional information about the phototransistor array 13 is described in U.S. patent application serial number 17 / 098,816, entitled “System and Method For Multi-Object Micro-Assembly Control With The Aid of A Digital Computer,” filed by Matei et al. on November 16, 2020, the disclosure of which is incorporated by reference.

[0039] In another embodiment, the array 35 of electrodes 12 may be programmable to generate the necessary sequence of potentials in a manner other than by using the phototransistor array 13, such as by using a memory-based array in which electrodes are activated via specific memory addresses.

[0040] Return to Figure 2, the system 10 also includes a high-speed camera 15 that tracks the position of the chiplets 11 being moved and can be used to estimate the chiplet 11 density distribution. In another embodiment, other types of sensors can be used to estimate and track the position of the chiplets 11 and the chiplet density distribution. Both the video projector 14 and the camera 15 are interfaced to one or more computing devices 16, which can control the electrodes 12 through the projector 14 to induce the formation of the desired chiplet density distribution. Other chiplet 11 patterns are also possible.

[0041] The connection between one or more computing devices 16 can be direct, such as via a wire or wireless connection to one or more computing devices 16 that are physically close, or the connection can reach one or more computing devices 16 that are physically remote via an interconnected network, such as via the Internet or a cellular network. One or more computing devices 16 include multiple computer processors that are specifically used to perform data processing in parallel. In one embodiment, the computer processor can be a graphics processing unit (GPU). In another embodiment, the computer processor can be a tensor processing unit (TPU) developed by Google, LLC in Mountain View, California, which is particularly dedicated to neural network machine learning. In another embodiment, the computer processor may include both GPU and TPU. In another embodiment, the computer processor may include other types of processors specifically used for parallel processing.

[0042] As described in further detail below, one or more computing devices 16 execute an optimization-based machine learning (ML) algorithm that uses a mathematical macro model and a cost function to calculate the chiplet density f(x, t) and a continuous function V(x, t) that assigns the electrode potential, where x is a 2D position and t is time. The computing device 16 shapes the electric field by the discrete geometry of the electrodes (e.g., a grid structure). Given the discretization of a continuous plane (i.e., an electrode grid), the computing device 16 can calculate the potential of a discrete point by integrating the potential field over a neighborhood of the discrete point based on a discretization scheme, where an example of such a discretization applied to the continuous potential function V(x, t) is referenced. Figure 4 See. Figure 4 is a diagram showing discrete grids superimposed on a continuous potential function, where the center of each grid represents an electrode 12 . Figure 4 The gray part refers to the low electrode potential value - Figure 4 The darker the grayscale color in a part of Figure 4 The lower the magnitude of the potential in that part. Figure 4 The dotted portion refers to the high potential value - Figure 4 The denser the dots in a part of Figure 4 The greater the magnitude of the electric potential in that part of the Automatic differentiation can be used to calculate. Then point x i (the center of the grid rectangle X i ) is calculated as

[0043] Although one or more computing devices 16 are shown as servers, other types of computer devices are possible. Computing device 16 may include one or more modules for executing the embodiments disclosed herein. Modules may be implemented as computer programs or procedures written as source code in conventional programming languages, and are presented for processors to execute as objects or byte codes. Alternatively, modules may also be implemented in hardware, as integrated circuits or burned into read-only memory components, and each computing device in computing device 16 may act as a dedicated computer. For example, when a module is implemented as hardware, the specific hardware is specifically used to perform the above-mentioned calculations and communications, and other computers cannot be used. In addition, when a module is burned into a read-only memory component, the computer storing the read-only memory becomes specifically used to perform operations that other computers described above cannot perform. Source code and various embodiments of objects and byte codes may be maintained on computer-readable storage media, such as floppy disks, hard drives, digital video disks (DVDs), random access memories (RAMs), read-only memories (ROMs), and similar storage media. Other types of modules and module functions are possible, as well as other physical hardware components. For example, computing device 16 may include other components found in programmable computing devices, such as input / output ports, network interfaces, and non-volatile storage, but other components are possible. In embodiments where computing device 16 is a server, the server may also be a cloud-based server or a dedicated server.

[0044] One or more computing devices 16 interface with storage device 17 and execute a capacitance modeler 18, which obtains parameters 19 of system 10, stores parameters 19 in storage device 17, and models the capacitance between electrode 12 and chiplet 11 and between chiplets 11 themselves. Parameters 19 may include the diameter of chiplet 11, the size of electrode 12, the dielectric fluid constant (e.g., such as ε=2), the fixed position of electrode 12, the material of chiplet 11 and electrode 12, and the vertical distance between chiplet 11 and electrode 12 (the "height" of chiplet 11). Other parameters 19 are still possible.

[0045] The capacity modeler 18 creates a capacitance-based dynamic model 20 for chiplet density. The capacity modeler 18 starts with a model 31 induced by a discrete (e.g., grid) electrode array 35. Next, the capacity modeler 18 goes from this discrete potential representation to a model 32 with a continuous potential representation, where the potential is assumed to be a continuum. The final step is to convert the continuous single chiplet model 32 into a chiplet density dynamic model 20 with the aid of the mean field approximation theory.

[0046] The following describes a 2D model 31 of the motion of a chiplet 11 under the influence of the potential field induced by the electrode array 12. The model is for only one chiplet 11 and omits possible interactions when the chiplets 11 are close to each other (where the exact distance a chiplet 11 needs to be kept away from another chiplet 11 in order to avoid causing interactions that need to be considered is determined based on the geometry of the micro-objects and the electrical potential - typically the distance between the two chiplets needs to be of the same magnitude as the distance from the chiplet 11 to the electrode 12 in order to avoid chiplet-chiplet interactions).

[0047] The result of applying an electric potential to the electrode 12 is the generation of a DEP force acting on the chiplet 11. The motion of the chiplet is opposed by a viscous drag force proportional to velocity, where the drag force is proportional to velocity in a non-turbulent flow, that is, when the Reynolds number is small. Due to the negligible mass of the chiplet, the acceleration is neglected. Therefore, the dynamic model of the motion of the chiplet 11 can be described by the following:

[0048]

[0049]

[0050] where (x, y) represents the chiplet position measured at the chiplet centroid, μ is the fluid-dependent viscosity coefficient, and F x (x, y) and F y (x, y) are the projections of the force induced by the potential field onto the x-axis and y-axis respectively. x and F y As a function of the potential energy of the chiplet 11. The capacity modeler 18 calculates the potential energy by using a capacitance-based circuit that generalizes the interaction between the electrode and the chiplet. Figure 5 Such a circuit is shown in which only one row of the array 35 having five electrodes 12 is depicted. Figure 5 It is shown that according to one embodiment Figure 2Figure 2 shows the capacitive-based interactions between a chiplet 11 and five electrodes of a system. The chiplet 11 and the electrodes 12 act as metal plates; therefore, the capacitance of these capacitors depends on the chiplet position. As expected, the maximum value is obtained when the chiplet 11 is positioned to maximize the overlap with the electrodes 12. To simplify the analysis, the analysis is limited to the low-frequency region, where the dielectric constant is not frequency-dependent.

[0051] Force F x and F y It can be expressed in the form of and where U(x, y) is the potential energy of chiplet 11, given by:

[0052]

[0053] Among them C i (x, y) is the capacitance between the chiplet and electrode i, V i is the potential of electrode i, V(x, y) is the potential of the chiplet assumed to be uniformly distributed on the chiplet, and N is the number of actuated electrodes. In the potential energy expression, the electrophoretic effect, which was found to be negligible, is neglected. By solving the reference Figure 5 The voltage and current in the circuit shown allow the chiplet potential to be calculated explicitly in terms of the electrode potentials. Specifically, the steady-state chiplet potential is given by:

[0054]

[0055] Feedback control design requires an explicit expression for the capacitance between the chiplet 11 and the electrode 12. The capacitance modeler 18 creates a model of the chiplet motion using a high-fidelity simulation using COMSOL ® developed by COMSOL, Inc. of Burlington, Massachusetts. software, but other types of simulations using other types of software are possible. For a symmetric chiplet (e.g., a bead), the capacitance is estimated by simulating a 2-dimensional electrostatic COMSOL model assuming that the electrodes are symmetric. This means that the capacitance function has the form C i (η)=C(||η-η i ||), where η T = [x, y] represents the chiplet location, and is the fixed position of electrode i.

[0056] Figure 6 is a diagram showing a COMSOL model of two conductors (spherical chiplet and electrode) for capacitance calculation according to one embodiment. Figure 6As shown, in the COMSOL model, a copper plate with a width of 16 μm and a thickness of 100 nm, and a spherical object of aluminum oxide (AlOx) with a diameter of 10 μm are surrounded by a dielectric with the same properties as the isopar-M solution.

[0057] The quasi-static model is calculated by the capacitance modeler 18 in the form of an electromagnetic simulation using partial differential equations, where we use a grounded boundary (zero potential) as the boundary condition. The capacitance matrix entries are calculated based on the charge that is generated on each conductor when a potential is applied to one of them and the other is set to ground. The COMSOL simulation does reflect the field distortion when the chiplet 11 approaches the electrode 12. The COMSOL electrostatic model has the following as parameters: the diameter of the sphere, the electrode size, the dielectric fluid constant (ε=2), and the position and material of the sphere and the electrode. The height of the chiplet 11 is fixed (z=5μm) and the position of the chiplet 11 on the x-axis is within the interval [-1mm, 1mm]. Note that due to the size of the chiplet 11 relative to the size of the electrode 12, the edge effect (electric field distortion at the edge) is significant. Offline simulations are performed for a range of chiplet positions: x∈[-1mm, 1mm] and z=5μm. The capacitance between the electrode and the chiplet 11 is evaluated for all considered positions. The capacitance function is parameterized using an error function: in is the error function, ξ is the distance between the center of the sphere and the center of the electrode assumed to be at the origin, and a i and c i >0 is a positive scalar, and δ is half the electrode spacing, ie 10 μm in this example.

[0058] Figure 7 A capacitance function fitted by training data generated by a COMSOL simulation is shown according to one embodiment. Figure 7 C(ξ), the capacitance between the chiplet and the electrode, is plotted as a function of the chiplet's horizontal position, where the values ​​are fitted to the error function parameterization. The capacity modeler 18 can be configured by using the transformation Mapping the 1D model to a 2D model yields the capacitance function For the spherical chiplet 11, it is sufficient to consider only one of the parameterizations of the capacitance function, and refer to Figure 8 The resulting 2D shape is depicted. Figure 8 is a graph showing a 2D capacitance function determined from a 1D model using symmetry according to one embodiment.

[0059] The intuitive rule behind the transition from model 31 with discrete representation to model 32 with continuous representation can be found in the expression of the potential of chiplet 11 Specifically, V(x) can be interpreted as the discrete random function V(Y) in the discrete distribution The expected value on the quality function p i(x) can be interpreted as the conditional probability pi (x) =Pr(Y=y i |X=x), and thus the chiplet 11 potential can be expressed as V(x)=E[V(Y)|X=x], where V(y) is a function reflecting the potential at each point y. Discrete probability distributions can be viewed as discretizations of continuous probability distributions, i.e. where y i is the position of the ith electrode and δ is half the electrode spacing. The parameterization of the capacitance function in terms of the error function tells us that the conditional probability density function (pdf) is a mixture of Gaussian functions. For the spherical shape example, the mixture has only one term and thus the capacitance is expressed as in in Therefore, the chiplet potential in the continuous representation can be expressed as where Y is a continuous random variable with a Gaussian distribution, i.e. and V(Y) is the function that assigns the electric potential to each point y. The potential energy can now be expressed as And thus the 1D chiplet dynamics is given by the following partial differential equation

[0060]

[0061] in and

[0062] The chiplet position is represented by x=(x1, x2) and the position of electrode i is represented by The chiplet dynamically becomes

[0063]

[0064] where μ is the viscosity coefficient and U(x) represents the potential energy of the chiplet. As in the 1D case, the potential energy is given by:

[0065]

[0066] Where V(x) represents the potential of the small chip, C i (x) = C(||xy (i) ||) represents the position y of the chiplet and electrode i (i) The capacitance between the i represents the potential of electrode i. Similar to the 1D case, assuming that the capacitance C ican be expressed as the unnormalized discretization of the multivariate Gaussian pdf, namely:

[0067]

[0068] in The potential energy is similar to the 1D case and is given by Given.

[0069] Consider a set of chiplets x i (t), where each chiplet 11 behaves according to (5), i.e. in Assume that the chiplet 11 to chiplet 11 interaction is negligible. The empirical distribution density associated with the solution x(t) of (5) is defined as:

[0070]

[0071] where δ is the Dirac delta function. Assume that the chiplet 11 remains in a fixed compact domain (e.g., an electrode array), for all N, and all t∈[0,T]. Therefore, for all t, is a P(R 2 ) in the probability measure. Therefore, according to Prohorov's theorem (described by Y. Prohorov, "Convergence of random processes and limit theorems in probability theory", Probability Theory and Its Applications, 1: 157-214, 1956, the disclosure of which is incorporated by reference), there exists a subsequence And f: R → R 2 , so that point by point converges to f, such as k converges to infinity. Next, derive the evolution equation satisfied by the limit metric. becomes a smooth test function. The inner product is represented by <·, ·> as follows:

[0072]

[0073] By consuming the time derivative of the left-hand side of the previous equation, as follows:

[0074]

[0075] therefore,

[0076]

[0077] For subsequence In the limit k→∞, the dynamic equation governing the variation of chiplet density over time is given by the partial differential equation (PDE):

[0078]

[0079] The previous PDE of equation (7) is set by the capacity modeler 18 as a dynamic model 20 for controlling chiplet density.

[0080] The one or more computing devices 16 further execute a position tracker 36 that analyzes the images 21 captured by the high-speed camera 15 (or data from another sensor) to estimate an existing (initial) density distribution 22 of the chiplets 11 - how densely the chiplets 11 are distributed through the various spatial regions within the housing. The position tracker 36 further receives (such as from a user) a desired target density distribution 23 - how densely the chiplets 11 should be distributed within the different disclosed spatial regions. The current chiplet density distribution 22, the target density distribution 20 are used by an MPC optimizer 25 executed by one or more of the computing devices 16 to determine a control scheme 24, which includes control inputs 27 - a sequence of electrical potentials that need to be generated by the electrodes 12 to induce the chiplets 11 to form a desired density. The MPC optimizer 25 executed by one or more computing devices 16 uses model predictive control (“MPC”) optimization to perform determinations, such as described in Camponogara et al., “Distributed model predictive control,” Control Systems, IEEE, 22(1):44-52, February 2002, and Garcia et al., “Model predictive control: Theory and practice—A survey,” Automatica, 25(3):335-348, 1989, the disclosures of which are incorporated by reference. MPC is a jump method for controlling nonlinear systems and is based on solving nonlinear programs using constraints.

[0081] One or more computing devices execute an MPC optimizer 25 that uses feedback control to shape a current density function (an initial density function describing the current density distribution 22 of the chiplets 11) f0(x) = f(x, 0) into a target density function f d(x) , which describes the target distribution 23 of the chiplets 11. The error between the current density function and the target density function is expressed using the Kullback-Leibler divergence, expressed as in is the domain of the density function. The goal of the MPC optimizer is to find a mapping of the potential distribution over time such that the initial chiplet density 22 is as close as possible to the target density distribution 23. The goal is to find a mapping V(x, t) that distributes the potential of each point x∈X and t∈[0, T] over time, where T is a certain time range. By adopting the model predictive control method, the ideal optimization problem that the MPC optimizer 25 needs to solve can be expressed as:

[0082]

[0083] Constrained by:

[0084]

[0085]

[0086] in is a compact set of possible potentials. The optimization problem searches for solutions to the partial differential equation (7) and the potential distribution function V(t, x) simultaneously. The first two constraints 26 enforce that f(t, x) is a solution to (7) and that it is a density function. The last constraint 26 defines the potential within some limits specified by the power supply. In its current form, the optimization problem of equation (8) is relatively intractable. To make the optimization problem tractable, a discretization of both X and the time domain [0, T] is considered. Specifically, consider the point set i∈0,...,n and Equation (8) can be reformulated as follows:

[0087]

[0088] Constrained by:

[0089]

[0090]

[0091] And which compares the continuous version of the density divergence with the discrete version of the density divergence, i.e. There are additional challenges: Assessment To compute the gradient of the potential function, the automatic differentiation features of a deep learning platform are utilized, such as described in: James Bradbury et al., “JAX: composable transformations of Python+NumPy programs,” 2018; Martin Abadi et al., “TensorFlow: Large-scale machine learning on heterogeneous systems,” 2015, software available from tensorflow.org; Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer, “Automatic differentiation in pytorch,” 2017, the disclosures of which are incorporated by reference.

[0092] To calculate the potential function, the MPC optimizer 25 needs to evaluate the variance of the multivariate, Gaussian conditional distribution, i.e. in Open and Where σ is a parameter determined based on the geometric properties of the chiplet assuming a spherical shape. To address this challenge, the MPC optimizer 25 uses a Gauss-Hermitian quadrature rule, such as described in Gene H. Golub and John H. Welsch, "Calculation of gauss quadrature rules", Technical Report, Stanford University, California, USA, 1967, the disclosure of which is incorporated by reference. The Gauss-Hermitian quadrature rule is commonly found in the theory of generalized chaotic polynomials (GPCs), such as described by: Anthony O'Hagan, "Polynomial chaos: A tutorial and critique from a statistician's perspective", Technical Report, University of Sheffield, UK, May 2013; Ralph C. Smith, "Uncertainty Quantification: Theory, Implementation, and Applications", Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 2013; Norbert Wiener, "The Homogeneous Chaos", American Journal of Mathematics, 60(4): 897-936, 1938; Dongbin Xiu, Didier Lucor, C.-H. Su, and George EmKamiadakis, "Performance evaluation of generalized polynomial chaos", Computational Science - ICCS 2003: International Conference, Melbourne, Australia and St. Petersburg, Russia, June 2-4, 2003, Proceedings, Part IV, pp. 346-354, Berlin, Heidelberg, 2003, SpringerBerlin Heidelberg, the disclosure of which is incorporated by reference. The Gauss-Hermitian quadrature rule provides a tool to efficiently evaluate conditional expectations. Thus, the expectation of a function of a random variable with a Gaussian distribution can be accurately approximated using the Gauss-Hermitian quadrature. Since the conditional probability distribution of Y|X can be expressed as the product of two Gaussian distributions where i∈1,2, so that the expectation of V(Y) is given by:

[0093]

[0094] Where n is the number of sample points, yi is the Hermitian polynomial H n(y) The physicist version of the root, and w i Reason The associated weights are given by . Similarly, the variance of V(Y) can be approximated as

[0095]

[0096] The final step in the effort to reach a tractable optimization formula is to parameterize both the solution f(x, t) of the PDE of equation (7) and the potential distribution map V(x, t). That is, the two smooth functions f(x, t) = f(t, x; θ f ) and V(x, t) = V(x, t; θ v ).pass Representing the combined parameterization, the final form of the optimization formula to be executed by the MPC optimizer is as follows:

[0097]

[0098] Constrained by:

[0099]

[0100]

[0101] The MPC optimizer can solve an optimization proxy of equation (12), where the PDE constraints and initial value boundary conditions are considered as weighted cost functions. Specifically, three loss functions are defined:

[0102]

[0103]

[0104]

[0105] And the overall loss function is defined as L(θ) = α1L1(θ) + α2L2(θ) + α3L3(θ), where α1, α2, α3 are non-negative scalars that assign different weights to the three loss functions. The density constraint is imposed by the definition. Specifically, given a function g that depends on θ: R×R 2 →R, the density function is parameterized as follows:

[0106]

[0107] Consider the box type of constraint 26 for the function V(x, t; β), that is The potential distribution function is defined as:

[0108]

[0109] For a smooth function Therefore, in practice, MPC Optimizer 25 solves the following optimization problem:

[0110]

[0111] There are four derivatives / gradients that need to be evaluated when solving the optimization problem of equation (16): and In one embodiment, the MPC optimizer 25 may utilize the JAX Python platform, as described in "JAX: composable transformations of Python+NumPy programs" by James Bradbury et al., 2018, which is incorporated by reference by its disclosure, which can automatically distinguish between native Python and NumPy functions, but in another embodiment, other types of software platforms may be utilized. In solving the optimization problem, the MPC optimizer 25 executes an Adam gradient-based optimization algorithm that is integrated with the JAX platform with automatic differentiation, as described by: Diederik P. Kingma and Jimmy Ba. Adam, "A method for stochastic optimization", 2014, cited in preprint: 1412.6980 Comment; published as a conference paper at the 3rd International Conference on Learning Representations, San Diego, 2015, the disclosure of which is incorporated by reference. Jax is a Python package that implements automatic differentiation, and additional information about it can be found at https: / / jax.readthedocs.io / en / latest / , the disclosure of which is incorporated by reference. Thus, the MPC optimizer 25 can utilize a machine learning algorithm (including automatic differentiation) to obtain the control input 27, i.e., the sequence of electrode potentials that will allow minimizing the difference between the existing chiplet density distribution 22 and the target chiplet density distribution 23.

[0112] The control input 27 generated by the MPC optimizer is used by a projector controller 28 executed by one or more computing devices 16 to map the control input 27 to an image 37 provided to a video projector 14, and the video projector in turn projects the image 37 to the phototransistor of the control electrode 11, thereby actuating the electrode 12 to induce movement of the chiplet 11. The mapping can be performed as described in: U.S. Patent Application Publication No. US20200207617, entitled “Micro-assembler system for manipulating micro-objects,” published on July 2, 2020 by Plochowietz et al., the disclosure of which is incorporated by reference.

[0113] In another embodiment, to increase the scalability of the process, the potential distribution map required to achieve the desired chiplet density may be derived based on the current chiplet distribution (rather than using MPC optimization).

[0114] Once the desired density distribution has been achieved, a system can be used to move individual particles as described in U.S. patent application serial number 17 / 121,411, entitled “System and Method for Machine-Learning Enabled Micro-Assembly Control With the Aid of a Digital Computer,” filed by Matei et al. on December 14, 2020, the disclosure of which is incorporated by reference.

[0115] The use of model predictive control to compute the potential distribution sequence that shapes the chiplet distribution into the desired pattern allows the desired chiplet distribution pattern to be achieved in a computationally efficient manner. In addition, the use of the Gauss-Hermitian quadrature rule to accurately approximate the desired, and the use of automatic differentiation to compute the derivatives of physical quantities (e.g., potential energy, chiplet distribution) and the gradient of the KL loss function provide additional tractability to the method. Fig. 9 is a flow chart showing a method 50 for micro-object density distribution control implemented by machine learning with the aid of a digital computer. The method 50 may be used Figure 2 The system is executed by a system. Parameters of the system are obtained, such as the diameter of the chiplets, the size of the electrodes, the dielectric fluid constant, the initial chiplet density distribution (which can be obtained using a high-speed camera), and the materials of the chiplets and electrodes, and the height of the chiplets, but other parameters are possible (step 51). A dynamic model that manages the change in chiplet density over time is set up (step 52), as described below with reference to Fig.10Further described and referenced above Figure 1 A target density distribution of chiplets is received (step 53). An MPC optimization is obtained for deriving a control input, which is a sequence of electrode potentials that will minimize the difference between the current chiplet density distribution and the target chiplet density distribution, as described above with reference to Figure 1 The control input is mapped to an image for display by the video projector (step 55), and the electrodes in the electrode array are controlled to induce movement of the chiplets to the desired position (step 56). If the desired density distribution has been achieved (as can be observed using the camera 15) and no further chiplet movement is required, the method 50 ends. If the target density distribution has not been achieved and the chiplets still need to be moved (step 57), the method 50 returns to step 54.

[0116] As mentioned above, in another embodiment, at least one sensor except the high-speed camera can be used for estimating density distribution. Likewise, in another embodiment, other modes for actuating programmable electrodes except the phototransistor array are possible. In these other embodiments, method 50 will be modified accordingly.

[0117] A capacitance-based model for dynamic control of chiplet density can be derived by using the mean-field approximation. Fig.10 is a diagram showing an arrangement according to an embodiment Fig. 9 A flow chart of a routine 60 for managing a dynamic model of chiplet density changes over time used in the method 50 is provided. Figure 1 As described, a simulation, such as a COMSOL simulation, is performed for the capacitance between the electrode and the chiplet at multiple distances (step 61). Based on the simulation, a discrete representation of a model for moving a chiplet in the chiplet based on the capacitance between the chiplet and the electrode is defined, as described above with reference to Figure 1 The discrete representation of the model is transformed into a continuous representation, as described above with reference to Figure 1 The mean field approximation is applied to the continuous representation of the model for dynamic control of chiplet density (step 64), and routine 60 ends.

[0118] For purposes of illustration and not limitation, two experimental examples of using the above-described systems and methods to manipulate chiplet density distribution are provided. In the examples, normalized representations of chiplet quantities are positioned on an electrode grid and potentials. They are similar to their physical counterparts by scaling factors. A 3×3 grid is used with a sampling period of 0.025. The chiplet capacitance is a function of the chiplet position x and the covariance matrix σ with mean and chiplet position x. 2IA multivariate Gaussian distribution of with σ = 0.05. A time range of 5 seconds and a sampling period of 0.1 seconds were considered. The PDE solution f(x, t) and the potential distribution function V(x, t) were modeled as a neural network with one hidden layer of size 30 and tanh as the activation function. Each function has an input of size 3 and an output of size 1. The step size of the Adam optimization algorithm is 0.001. The number of epochs varies based on the quality of the solution. By using Gauss-Hermitian quadrature, 30 points are used to approximate the expectations in equations (10) and (11) (where the points are represented by the term "n" in these equations), which results in a total of 30×30 points.

[0119] In use Figure 2 systems and Fig. 9 In the first example of implementing a chiplet density distribution change by the method, the goal is to start with a "flat" distribution of chiplets 11 and generate a potential distribution sequence of chiplets concentrated around the origin. The initial distribution is a 2D Gaussian distribution with zero mean and covariance matrix σ 2I , where σ=2. FIG. 11A to FIG. 11F is a graph showing the chiplet density distribution of the first example at different times. The chiplet distribution starts at t=0 seconds with an almost flat distribution and achieves the desired distribution at t=5 seconds. FIG. 12A to FIG. 12F Shows the method used to create FIG. 11A to FIG. 11F Evolution of the potential of the electrodes with a small chip density distribution over time. FIG. 12A to FIG. 12F The grayscale portions of the graph refer to low electrode potential values ​​- the darker the grayscale color in a portion of the graph, the lower the potential in that portion of the graph. FIG. 12A to FIG. 12F The stippled portions of the graph refer to high potential values ​​- the denser the stippling in a portion of the graph, the greater the potential value in that portion of the graph. FIG. 12A to FIG. 12F In this example, the x-axis and y-axis are the first (x1) and second (x2). FIG. 13A to FIG. 13F The evolution of the force vector field is shown. FIG. 13A to FIG. 13F is shown for creating FIG. 11A to FIG. 11F A plot of the force vector field over time for the density distribution of chiplets. FIG. 13A to FIG. 13F In the figure, the arrow at a point indicates the direction in which the chip at that point is pushed, and the color intensity is proportional to the force value.

[0120] In use Figure 2 systems and Fig. 9 In the second example of changing the density distribution of small chips by using the method of , the small chips are divided into two piles. The example starts with a zero mean, a Gaussian distribution covariance matrix σ 2I , where σ = 2, and two separate piles are created. The final distribution is a mixture of two Gaussian distributions, where the two distributions are and Both distributions have a covariance matrix σI, where σ = 0.5. FIG. 14A to FIG. 14F The evolution of the chiplet distribution over time is shown. FIG. 14A to FIG. 14F is a graph showing the chiplet density distribution of the second example at different times. At t=5 seconds, the desired chiplet separation is achieved. FIG. 15A to FIG. 15F Shows the method used to create FIG. 14A to FIG. 14F Evolution of the potential of the electrodes with a small chip density distribution over time. FIG. 15A to FIG. 15F The grayscale portions of the graph refer to low electrode potential values ​​- the darker the grayscale color in a portion of the graph, the lower the potential in that portion of the graph. FIG. 15A to FIG. 15F The stippled portions of the graph refer to high potential values ​​- the denser the stippling in a portion of the graph, the greater the potential value in that portion of the graph. FIG. 15A to FIG. 15F , the x-axis and y-axis are the first (x1) and the second (x2). FIG. 16A to FIG. 16F is shown for creating FIG. 14A to FIG. 14F A plot of the force vector field over time for the density distribution of chiplets. FIG. 16A to FIG. 16F In the figure, the arrow at a point indicates the direction in which the chip at that point is pushed, and the color intensity is proportional to the force value. FIG. 16A to FIG. 16F Confirm the intuitive rules about how the field should look. At t=0 seconds, there is a force pushing outward from the center. When the chiplet reaches its final distribution, the force surrounds both regions, pushing inward.

[0121] While the invention has been particularly shown and described with reference to embodiments thereof, it will be understood by those skilled in the art that the foregoing and other changes in form and details may be made therein without departing from the spirit and scope of the invention.

Claims

1. A system for micro-object density distribution control by means of a digital computer, comprising: One or more processors configured to execute computer executable code, the processors configured to: obtaining one or more parameters of a system for locating a plurality of micro-objects, the system comprising a plurality of programmable electrodes configured to induce movement of the micro-objects when the micro-objects are in proximity to the electrodes after one or more electrical potentials are generated by one or more of the electrodes; defining a density model describing changes in density of micro-objects over time due to capacitance-based interactions based on the parameters of the system; estimating a density distribution of the micro-objects using at least one sensor; receiving a target density distribution of the micro-objects; performing a model predictive control (MPC) optimization to derive a sequence of potentials to be generated by at least some of the electrodes based on the density model, the sequence of potentials being used to move at least some of the micro-objects to minimize an error between the estimated density distribution and the target density distribution; At least some of the electrodes are actuated to generate the sequence of the potentials.

2. The system according to claim 1, further comprising: defining a discrete representation of a movement model for moving one of the micro-objects based on the parameter; transforming the discrete representation of the motion model into a continuous representation; as well as A mean field approximation is applied to the continuous representation of the motion model to obtain the density model.

3. The system according to claim 2, further comprising: performing a plurality of simulations of the capacitance between the electrode and one of the micro-objects; as well as A function included in the movement model is defined, and the capacitance between the micro-object and each of the electrodes is described as a function of the distance between the micro-object and the electrode.

4. The system of claim 3, wherein the movement model describes the movement in at least one of one dimension and two dimensions. 5 . The system of claim 1 , wherein the error between the estimated density distribution and the target density distribution is expressed using Kullback-Leibler divergence.

6. The system of claim 1, wherein performing the MPC optimization comprises performing automatic differentiation to compute a plurality of gradients.

7. The system of claim 1, wherein performing the MPC optimization comprises evaluating at least one expectation using Gauss-Hermitian quadrature. 8 . The system of claim 1 , wherein the processor comprises at least one of one or more graphics processing units (GPUs) and one or more tensor processing units (TPUs).

9. The system of claim 8, wherein two or more of the processors work in parallel to perform the MPC optimization.

10. The system of claim 1, wherein the electrodes are controlled by phototransistors, the one or more processors being further configured to: mapping the potentials in the sequence to a plurality of images; A video projector is controlled to project the image to the phototransistor, wherein the phototransistor controls the electrode to generate the sequence of potentials based on the projected image.

11. A method for micro-object density distribution control by means of a digital computer, comprising: obtaining, by one or more processors configured to execute computer executable code, one or more parameters of a system for locating a plurality of micro-objects, the system comprising a plurality of programmable electrodes configured to induce movement of the micro-objects when the micro-objects are in proximity to the electrodes after one or more electrical potentials are generated by one or more of the electrodes; defining, by one or more of the processors based on the parameters of the system, a density model describing changes in density of micro-objects over time due to capacitance-based interactions; estimating, by one or more of the processors, a density distribution of the micro-objects using at least one sensor; receiving, by one or more of the processors, a target density distribution of the micro-objects; performing, by one or more of the processors, a model predictive control (MPC) optimization to derive, based on the density model, a sequence of potentials to be generated by at least some of the electrodes, the sequence of potentials being used to move at least some of the micro-objects so as to minimize an error between the estimated density distribution and the target density distribution; At least some of the electrodes are actuated by one or more of the processors to generate the sequence of potentials.

12. The method according to claim 11, further comprising: defining a discrete representation of a movement model for moving one of the micro-objects based on the parameter; transforming the discrete representation of the motion model into a continuous representation; as well as A mean field approximation is applied to the continuous representation of the motion model to obtain the density model.

13. The method according to claim 12, further comprising: performing a plurality of simulations of the capacitance between the electrode and one of the micro-objects; as well as A function included in the movement model is defined, and the capacitance between the micro-object and each of the electrodes is described as a function of the distance between the micro-object and the electrode.

14. The method of claim 13, wherein the movement model describes the movement in at least one of one and two dimensions.

15. The method of claim 11, wherein the error between the estimated density distribution and the target density distribution is expressed using Kullback-Leibler divergence.

16. The method of claim 11, wherein performing the MPC optimization comprises performing automatic differentiation to compute a plurality of gradients.

17. The method of claim 11, wherein performing the MPC optimization comprises evaluating at least one expectation using Gauss-Hermitian quadrature. 18 . The method of claim 11 , wherein the processor comprises at least one of one or more graphics processing units (GPUs) and one or more tensor processing units (TPUs).

19. The method of claim 18, wherein two or more of the processors operate in parallel to perform the MPC optimization.

20. The method of claim 11, wherein the electrodes are controlled by phototransistors, the one or more processors being further configured to: mapping the potentials in the sequence to a plurality of images; A video projector is controlled to project the image to the phototransistor, wherein the phototransistor controls the electrode to generate the sequence of potentials based on the projected image.

Images