Artificial intelligence-based cell injection instrument motion intelligent control system

Abstract

The present application relates to cell injection instrument motion control technical field, especially based on the cell injection instrument motion intelligent control system of artificial intelligence, its technical scheme includes: through the construction multimodal perception fusion module and cell mechanics characteristic on-line identification module, fundamentally solved the problem of poor universality and high risk of injury caused by biological sample diversity; The introduction of adaptive motion planning and decision-making based on reinforcement learning center, the motion control problem is converted into a continuous learning optimization process; The designed feedforward and feedback composite execution module, especially its built-in iterative learning controller, effectively solves the bottleneck that the repeatability error is difficult to eliminate in the precision motion system; The integrated abnormal state autonomous diagnosis and recovery module gives the system high robustness and fault tolerance ability; Through the parallel real-time pattern recognition, the system can quickly and accurately diagnose a variety of common operation faults, and trigger intelligent recovery strategy, greatly reduce the dependence on artificial intervention.

Description

Technical Field

[0001] This invention relates to the field of motion control technology for cell injection devices, and more particularly to an intelligent motion control system for cell injection devices based on artificial intelligence. Background Technology

[0002] In the field of precision instruments and automation control, achieving precise motion control at the micron and even nanometer level is one of the core technological challenges, and it is widely used in cutting-edge fields such as biomedicine, microelectronics manufacturing, and materials science. Among them, cell injection, as a key biomanipulation technology, aims to accurately, efficiently, and non-destructively inject exogenous substances (such as genes, drugs, or tracers) into target cells through high-precision mechanical motion.

[0003] Among these technologies, automated motion control of cell injectors is a key area for improving injection success rates and operational efficiency. This technology aims to drive the injection needle to complete a series of complex actions, such as cell positioning, puncture, and injection, through preset or real-time adjusted motion trajectories and parameters. This replaces the traditional manual injection mode that relies on manual operation of the microscope, thereby overcoming the instability of results and low throughput bottlenecks caused by differences in operator skills.

[0004] Existing technologies typically employ rigid motion control strategies based on pre-programmed procedures, or combine them with simple visual feedback for position correction. However, these methods have significant limitations: First, their control models struggle to adapt to the inherent diversity and dynamism of biological samples. For example, variations in mechanical properties caused by differences in cell types, sizes, morphologies, and culture substrates make fixed puncture force or velocity parameters prone to cell damage or injection failure. Second, existing systems lack in-depth fusion analysis and intelligent reasoning capabilities for the multimodal data (such as microscopic image sequences, pressure sensor signals, and displacement feedback) generated in real time during injection, making it impossible for them to autonomously learn from the data and dynamically optimize control strategies. Furthermore, facing the high-throughput injection demands of cell populations, existing control logic exhibits insufficient adaptability in task planning, path optimization, and anomaly handling (such as needle tip blockage and cell drift), heavily relying on manual intervention and limiting the overall automation level and reliability of the operation. Therefore, how to construct an intelligent control system capable of sensing complex operating environments, making autonomous decisions, and optimizing motion behavior in real time has become a core challenge that urgently needs to be addressed to improve the efficiency of cell injection technology. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent motion control system for a cell injection device based on artificial intelligence, in order to solve the problems of high cell damage rate, unstable injection success rate and limited automation level caused by rigid control models, insufficient multimodal data fusion and analysis capabilities, and lack of adaptive ability when facing the diversity and dynamism of biological samples in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] The AI-based intelligent motion control system for cell injection devices includes a multimodal perception fusion module, an online cell mechanical property identification module, an adaptive motion planning and decision-making center, a feedforward and feedback composite execution module, and an abnormal state autonomous diagnosis and recovery module.

[0008] Among them, the multimodal perception fusion module collects and processes microscopic visual data, force sensing data, displacement data and environmental state data from the cell injection instrument operating environment in real time and synchronously, and generates a fusion perception feature tensor under a unified spatiotemporal reference.

[0009] The online cell mechanical property identification module receives the fused sensing feature tensor from the multimodal sensing fusion module. Through the built-in deep neural network model, it calculates the equivalent elastic modulus, membrane tension threshold, and viscous damping coefficient of the current target cell in real time and outputs a dynamic vector of cell mechanical property parameters.

[0010] The adaptive motion planning and decision-making center receives the cell mechanical property parameter vector output by the cell mechanical property online identification module, and combines it with the preset injection task objective to generate a real-time, optimized three-dimensional motion trajectory sequence of the injection needle, axial feed velocity curve, and puncture force threshold curve based on the reinforcement learning framework.

[0011] The feedforward and feedback composite execution module receives motion commands from the adaptive motion planning and decision-making center, drives the piezoelectric ceramic actuator and nano-positioning platform to perform motion, and at the same time, through the built-in iterative learning controller, performs online compensation and learning for trajectory tracking errors during the execution process to ensure the consistency between actual motion and commanded motion.

[0012] The abnormal state autonomous diagnosis and recovery module continuously monitors the raw data stream of the multimodal perception fusion module and the state feedback of the feedforward and feedback composite execution module. It uses pattern recognition algorithms to detect abnormal events such as needle tip blockage, unexpected cell displacement, and culture medium interface disturbance in real time, and triggers predefined or dynamically generated recovery strategies by the decision center.

[0013] Furthermore, the multimodal perception fusion module includes a digital microscope camera, a nanometer-resolution capacitive force sensor, a grating ruler displacement encoder, and a temperature and humidity sensor. Internally, the multimodal perception fusion module deploys a feature extraction and synchronization unit. This unit performs real-time cell edge and pinpoint sub-pixel localization on the microscopic image sequence, performs noise reduction and feature extraction on the force sensor signal, performs differential calculation on the displacement encoder signal to obtain real-time velocity and acceleration, and aligns the timestamps of all data streams to the system master clock. The output of the feature extraction and synchronization unit is a four-dimensional tensor, with dimensions corresponding to the time series, spatial location, sensor type, and feature channel, respectively.

[0014] Furthermore, the deep neural network model built into the online cell mechanical property identification module is a cascaded deep neural network model. The first level of the cascaded deep neural network model is a convolutional neural network branch, which processes the microscopic image features from the multimodal perception fusion module to predict the cell morphology and apparent stiffness. The second level of the cascaded deep neural network model is a temporal recurrent neural network branch, which processes the micro-force and displacement temporal feature pairs from the multimodal perception fusion module to fit the force and displacement response curves of the cell in the initial stage of microneedle contact. The third level of the cascaded deep neural network model is a fully connected fusion network, whose input is the output feature vector of the first and second level networks. After multiple nonlinear transformations, it directly regresses and outputs a cell mechanical property parameter vector containing three scalar parameters.

[0015] Furthermore, the reinforcement learning framework upon which the adaptive motion planning and decision-making center is based is the proximal policy optimization reinforcement learning framework. The proximal policy optimization reinforcement learning framework models the cell injection process as a Markov decision process. The state space of the Markov decision process is defined as the cell mechanical property parameter vector at the current moment, the six-degree-of-freedom pose of the injection needle tip relative to the target cell, and the historical motion trajectory fragments. The action space of the Markov decision process is defined as the displacement increment of the injection needle in three-dimensional space, the axial feed velocity command, and the maximum allowable contact force command in the next control cycle. The reward function of the Markov decision process consists of a weighted sum of multiple sub-rewards, including a successful puncture reward, a cell morphology integrity preservation reward, an operation time penalty, and a motion smoothness reward.

[0016] Furthermore, the adaptive motion planning and decision-making center includes a policy network and a value network; the policy network outputs the probability distribution of actions based on the current state; the value network evaluates the long-term cumulative reward expectation of the current state; during each injection task execution, the state-action-reward sequence collected by the system is used to adjust the parameters of the policy network and the value network online.

[0017] Furthermore, the feedforward and feedback composite execution module includes a piezoelectric ceramic actuator, a nano-positioning platform, a digital-to-analog converter and power amplifier circuit, and an iterative learning controller. The iterative learning controller is based on the feedforward and feedback composite control. Its feedback loop adopts a proportional-integral-derivative controller to suppress random disturbances. Its feedforward loop adopts model-based inverse dynamics compensation to offset the known nonlinearity and hysteresis characteristics of the system.

[0018] Furthermore, the iterative learning controller has a built-in error memory and learning unit; the error memory and learning unit records the history of trajectory tracking errors when performing motion trajectories for the same type of cells in the current injection task; in the subsequent motion phase, the error memory and learning unit generates a feedforward compensation signal based on the historical error data, which is directly superimposed on the control command.

[0019] Furthermore, the abnormality detection of the abnormality autonomous diagnosis and recovery module is based on a set of parallel dedicated classifiers. For needle tip blockage abnormalities, the classifier analyzes the force sensor signal and steady-state resistance value in real time and compares it with the historical baseline of the clean needle tip. When the resistance continues to exceed the threshold and the vibration energy decreases, it is determined to be a blockage. For unexpected cell displacement abnormalities, the classifier compares the centroid position of the target cell in consecutive frame microscopic images and performs motion compensation calculations by combining the feature points of the culture medium substrate. When a rigid displacement or deformation of the cell exceeding the preset range is detected, it is determined to be a displacement abnormality.

[0020] Furthermore, once the abnormal state autonomous diagnosis and recovery module detects a confirmed abnormal event, it immediately sends an interrupt signal and an abnormality type code to the adaptive motion planning and decision-making center. The adaptive motion planning and decision-making center then calls the corresponding recovery protocol from the preset strategy library according to the abnormality type, or plans a new safe withdrawal trajectory and relocation trajectory in real time based on the current environmental state, and instructs the feedforward and feedback composite execution module to execute it.

[0021] Furthermore, the system operates within a hierarchical real-time control architecture, which includes a task management layer, an intelligent decision-making layer, and a real-time control layer. The task management layer operates on a second-level timescale and is responsible for task parsing, batch planning, and human-computer interaction. The intelligent decision-making layer operates on a millisecond-level timescale and encompasses the functions of adaptive motion planning, a decision-making center, and an abnormal state autonomous diagnosis and recovery module, performing online decision-making and planning. The real-time control layer operates on a microsecond-level timescale and includes the underlying control loop of the feedforward and feedback composite execution module, ensuring the execution of instructions. The three layers exchange data through a deterministic real-time communication bus.

[0022] Compared with the prior art, the beneficial effects of the present invention are:

[0023] 1. This invention, by constructing a multimodal perception fusion module and an online cell mechanical property identification module, has for the first time achieved real-time quantitative perception and calculation of the individualized and dynamic mechanical properties of living cells. The system abandons the fixed parameter model and can adaptively adjust the control strategy according to the real-time response of each cell, fundamentally solving the problems of poor universality and high risk of damage caused by the diversity of biological samples, and upgrading cell injection operation from experience-based "blind operation" to data-based "perception operation".

[0024] 2. This invention introduces an adaptive motion planning and decision-making center based on reinforcement learning, transforming the motion control problem into a continuous learning optimization process. In actual operation, this system can autonomously learn the optimal injection strategy through interaction with the environment, not only optimizing the success rate and cell survival rate of a single injection, but also achieving comprehensive optimization of multiple objectives such as operation speed and smoothness through the guidance of the reward function. This combination of data-driven and model-driven methods significantly improves the system's intelligence and autonomous decision-making ability when facing complex and unstructured operating environments.

[0025] 3. The feedforward and feedback composite execution module designed in this invention, especially its built-in iterative learning controller, effectively overcomes the bottleneck of the difficulty in eliminating repeatability errors in precision motion systems. The controller can actively learn and compensate for the inherent nonlinearity and hysteresis characteristics of the system by utilizing historical execution data, so that the motion control accuracy can be self-evolved and improved with the accumulation of operation times, providing a reliable technical guarantee for achieving and maintaining submicron or even nanometer-level positioning and tracking accuracy in the long term.

[0026] 4. The abnormal state autonomous diagnosis and recovery module integrated in this invention endows the system with high robustness and fault tolerance. Through parallel real-time pattern recognition, the system can quickly and accurately diagnose a variety of common operational faults and trigger intelligent recovery strategies, which greatly reduces the reliance on manual intervention. This not only improves the reliability and efficiency of high-throughput continuous operation, but also reduces the risk of sample loss due to operation interruption or failure, realizing fully automatic and unattended cell injection operation. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the overall technical solution architecture of the cell injection device motion intelligent control system based on artificial intelligence proposed in this invention;

[0028] Figure 2 This is a schematic diagram of the core principle framework of the online cell mechanical property identification module in this invention;

[0029] Figure 3 This is a logical flowchart of the adaptive motion planning and decision-making center in this invention;

[0030] Figure 4 This is a schematic diagram of the multi-level interaction relationship and data flow of the feedforward and feedback composite execution module in this invention;

[0031] Figure 5 This is a schematic diagram illustrating the interaction and recovery strategy triggering between the abnormal state autonomous diagnosis and recovery module and other parts of the system in this invention. Detailed Implementation

[0032] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0033] Example 1:

[0034] This invention provides an artificial intelligence-based intelligent control system for a cell injection device. For example... Figure 1 As shown, the system physically implements a hierarchical real-time control architecture, comprising a task management layer, an intelligent decision-making layer, and a real-time control layer from top to bottom. The task management layer operates on a second-level timescale, and its core function is to parse the injection task parameters input by the user through the human-computer interaction interface. These parameters include, but are not limited to, target cell type, expected injection site, injection substance volume, and the number and arrangement of cells in the batch. Based on these parameters, the task management layer generates macroscopic task sequences and logical flows, and issues task instructions and status query requests to the lower layers via a deterministic real-time communication bus. The intelligent decision-making layer operates on a millisecond-level timescale and is the core carrier of the system's intelligence. It receives macroscopic instructions from the task management layer and processes data streams from the perception layer in real time, performing online decision-making, planning, and anomaly diagnosis. The real-time control layer operates on a microsecond-level timescale, directly driving the physical actuators and ensuring that the motion instructions issued by the intelligent decision-making layer are executed with high precision and high bandwidth. The three layers exchange data via a time-triggered real-time Ethernet bus, with the bus cycle strictly synchronized to the system's master clock, ensuring deterministic low latency and high reliability in the transmission of status information and instructions.

[0035] The system's perception and decision-making execution functions are achieved collaboratively through five core modules. The first core module is the multimodal perception fusion module. As the system's perception front-end, this module is responsible for real-time acquisition and synchronous processing of multi-source heterogeneous sensor data from the cell injection instrument's operating environment. Its hardware integration includes a high-speed digital microscope camera with a frame rate of at least 2000 frames per second, equipped with a high numerical aperture objective lens and a long working distance condenser lens, capable of providing bright-field or phase-contrast microscopic image sequences with sub-micron spatial resolution; a nanometer-resolution capacitive microforce sensor with a measurement range of ±100 microNewtons and a resolution better than 10 nanoNewtons, directly integrated inside the needle holder, used to measure the three-dimensional contact force during the contact between the needle tip and the cells and culture medium; a grating ruler displacement encoder system, including three orthogonal grating rulers and a readhead, with a measurement resolution of 0.1 nanometers, used to provide feedback on the absolute position of the injection needle drive platform in three-dimensional space; and a set of environmental status sensors, including temperature and humidity sensors and a carbon dioxide concentration sensor integrated near the culture dish, used to monitor the stability of the cell culture environment. The multimodal perception fusion module internally deploys a dedicated feature extraction and synchronization unit, which is implemented collaboratively by a field-programmable gate array (FPGA) and a digital signal processor (DSP). For microscopic image sequences, the feature extraction and synchronization unit executes a real-time image processing pipeline. First, Gaussian filtering is performed for noise reduction. Then, the Canny edge detection algorithm combined with sub-pixel interpolation technology is used to accurately extract the contour edges of the target cells and the sub-pixel coordinates of the injection needle tip. The unit calculates the cell's centroid position, equivalent diameter, roundness, and the distance and angle of the needle tip relative to the cell's centroid. For micro-force sensor signals, the unit first performs hardware synchronous acquisition at a sampling rate of 100 kHz. Then, a fourth-order Butterworth low-pass filter is used to filter out high-frequency electrical noise, and the force signal components, resultant force magnitude, gradient, and vibration energy spectrum in a specific frequency band, such as 100 Hz to 1000 Hz, are calculated in real time. For grating ruler displacement encoder signals, the unit performs high-speed differential calculations to obtain the real-time velocity and acceleration of the injection needle in three directions. Before entering the processing pipeline, all data streams are marked with a precise timestamp originating from the same system's master clock, with a timestamp accuracy of 1 microsecond. The feature extraction and synchronization unit ultimately aligns and encapsulates all processed features according to a unified spatiotemporal reference, outputting a structured four-dimensional fusion sensing feature tensor. The first dimension of this tensor is the time-series index, corresponding to consecutive sampling times. The second dimension is the spatial grid index, dividing the field of view into a 32×32 grid, with each grid storing the features of that region. The third dimension is the sensor type index, corresponding to visual features, force features, displacement features, and environmental features, respectively. The fourth dimension is the feature channels; for example, for visual feature channels, this may include scalar values ​​for a total of 128 channels, such as edge intensity, grayscale gradient, and texture features.This four-dimensional tensor is pushed to downstream modules in real time via a high-speed data bus.

[0036] The second core module is the online identification module for cell mechanical properties. For example... Figure 2As shown, this module receives fused sensing feature tensors from the multimodal sensing fusion module. Its core task is to calculate the individualized and dynamic mechanical property parameters of the current target cell in real time. The core of the module is a pre-trained cascaded deep neural network model deployed on a dedicated graphics processor in the intelligent decision layer to ensure millisecond-level inference speed. This model consists of three cascaded sub-networks. The first level is a convolutional neural network branch whose input is visually relevant feature slices from the fused sensing feature tensor. This branch contains 5 convolutional layers and 3 pooling layers, with the kernel size decreasing from 7×7 to 3×3 and the number of channels increasing from 64 to 256. This branch is specifically used to extract high-level morphological and textural features from the cell's microscopic image and outputs a 256-dimensional feature vector. This vector encodes the cell's morphological category information and prior information on apparent stiffness, such as whether the cell belongs to a fibroblast, neuron, or oocyte, and whether its membrane surface is wrinkled or smooth. The second level is a temporal recurrent neural network branch, specifically employing a long short-term memory network structure. The input is a feature sequence related to the microforce-displacement temporal pair in the fused sensing feature tensor, typically consisting of data from the last 100 sampling times. This Long Short-Term Memory (LSTM) network contains two hidden layers, each with 128 units. It is specifically designed to dynamically model the dynamic response relationship between force and displacement exhibited by the cell in the initial stage of needle tip contact, learning its viscoelastic behaviors such as creep and relaxation. This branch outputs a 128-dimensional feature vector encoding the cell's dynamic mechanical response pattern. The third level is a fully connected fusion network, whose input is a 384-dimensional vector formed by concatenating the 256-dimensional feature vector output from the first-level convolutional neural network branch and the 128-dimensional feature vector output from the second-level LSM network branch. This fully connected network contains three hidden layers with 256, 128, and 64 neurons respectively, each followed by a batch normalization layer and a ReLU activation function. The final output layer is a linear layer that directly regresses and outputs a vector of cell mechanical property parameters containing three scalar parameters. These three parameters have explicit physical meanings and units. The first parameter is the cell's equivalent elastic modulus, measured in kilopascals, which characterizes the cell's ability to resist elastic deformation. The second parameter is the cell's membrane tension threshold, measured in nanonewtons per micrometer, which characterizes the maximum tension the cell membrane can withstand before rupture. The third parameter is the cell's viscous damping coefficient, measured in nanonewtons per micrometer, which characterizes the energy dissipation characteristics caused by cytoplasmic flow and membrane fluidity within the cell. This deep neural network model must undergo extensive offline training before system deployment. The training dataset consists of numerous experiments, including standard polyacrylamide microspheres with known mechanical parameters, as well as various types of real cells such as HeLa cells, MCF-7 cells, and mouse oocytes. Accurate true values ​​of the mechanical parameters for these samples were obtained using standard mechanical testing methods such as microtubule aspiration and atomic force microscopy nanoindentation.Simultaneously, multimodal sensing data of these samples were collected under simulated injection conditions, including images, force curves, and displacement curves, forming an input-output paired dataset. Training employed a mean squared error loss function and the Adam optimizer, undergoing over 1 million iterations to enable the model to accurately regress the cell's mechanical properties from the multimodal sensing data. During online operation, the module received the latest fused sensing feature tensor at a frequency of 1000 times per second and performed forward inference, outputting a dynamically updated vector of cell mechanical property parameters, providing crucial information for motion planning.

[0037] The third core module is the adaptive motion planning and decision-making center. For example... Figure 3 As shown, this module is the core decision engine of the intelligent decision layer. It receives the cell mechanical property parameter vector output in real time from the online cell mechanical property identification module and, combined with the injection task objective issued by the task management layer, generates the optimal injection needle motion command. Its operating mechanism is based on a proximal policy optimization reinforcement learning framework, formalizing the entire cell injection process as a Markov decision process. The mathematical model of this process is as follows: The system's state space S is a high-dimensional continuous vector. At each decision time t, the state st is specifically defined as the concatenation of three parts of information. The first part is the cell mechanical property parameter vector at the current time, namely the equivalent elastic modulus Et, membrane tension threshold Tt, and viscous damping coefficient Ct. The second part is the six-degree-of-freedom pose of the injection needle tip relative to the target cell, including three position coordinates xt, yt, zt and three Euler angle poses θxt, θyt, θzt. This information is calculated by fusing visual positioning and displacement encoder data provided by the multimodal perception fusion module. The third part is the historical motion trajectory segment, namely the velocity and acceleration sequence of the injection needle in the past 10 decision cycles. The system's action space A is also a continuous space. At each decision time t, the action at output by the agent, i.e., the decision center, is a 7-dimensional vector. The first three dimensions define the displacement increments Δx, Δy, and Δz of the injection needle in the three-dimensional task space within the next control cycle, in micrometers. The fourth dimension defines the feed velocity command vz of the injection needle along its axis, in micrometers per second. Dimensions 5 to 7 define the maximum permissible contact force commands Fmaxx, Fmaxy, and Fmaxz in the three coordinate axes, in nanonewtons, used to protect cells from excessive lateral force damage. The system's reward function R is key to driving the agent's learning; it consists of a weighted sum of multiple carefully designed sub-rewards. Specifically, the reward function is defined as:

[0038] R(t)=w1*R(puncture)+w2*R(integrity)+w3*R(time)+w4*R(smoothness)

[0039] Among them, R (puncture) is the reward for successful puncture; a large positive reward is given when the needle tip detects a breakthrough of the cell membrane and the micro-force sensor shows a sudden drop in force. R (integrity) is the reward for maintaining cell morphological integrity; a continuous positive reward is given if the rate of change of cell contour roundness is below a threshold, and a negative reward is given if the cell undergoes drastic deformation. R (time) is the penalty for operation time; a small negative reward is given for each decision cycle to encourage rapid completion of the operation. R (smoothness) is the reward for motion smoothness; the smaller the change in the second norm of the action in adjacent cycles, the higher the reward, encouraging the generation of smooth motion trajectories. The weight coefficients w1, w2, w3, and w4 were determined through offline simulation debugging, for example, set to 10, 2, -0.01, and 0.5 respectively. The adaptive motion planning and decision-making center contains two neural networks: a policy network πθ and a value network Vφ. The policy network πθ takes the current state st as input, processes it through three fully connected layers with 256 neurons each, and outputs the mean μ and variance σ of each dimension of the action at, thus defining a Gaussian probability distribution. The agent samples according to this distribution to obtain the final action to be executed. The value network Vφ also takes the state st as input and evaluates the expected value of the cumulative discounted reward that can be obtained in the future when in that state. At the initial deployment of the system, the policy network and the value network have been pre-trained for millions of rounds in a high-fidelity physical simulation environment, mastering basic injection skills. In actual operation, after each injection task or decision segment is completed, the system collects a series of state, action, and reward sequences. These sequences are fed into the proximal policy optimization algorithm for online fine-tuning. The algorithm safely updates the parameters θ and φ of the policy network πθ and the value network Vφ by calculating the advantage function and imposing constraints on the policy update magnitude. This allows the system's motion planning strategy to continuously and adaptively optimize according to factors such as cell diversity and environmental disturbances encountered in actual operation, constantly approaching the optimal control strategy for the current specific sample.

[0040] The fourth core module is the feedforward and feedback composite execution module. For example... Figure 4As shown, this module belongs to the real-time control layer and is responsible for converting abstract motion commands issued by the adaptive motion planning and decision-making center into high-precision physical motion. Its hardware actuators include a set of high-bandwidth piezoelectric ceramic actuators with a stroke range of 100 micrometers and a closed-loop bandwidth greater than 2 kHz. A nano-positioning platform based on a flexible hinge has a motion range of 100 micrometers × 100 micrometers × 100 micrometers in all three degrees of freedom, achieving a resolution of 0.3 nanometers. It also includes a matching high-precision 18-bit digital-to-analog converter and a linear power amplifier to drive the piezoelectric ceramic actuators. The core of this module's control is an iterative learning controller, based on a feedforward and feedback composite control architecture. The feedback control uses a digital proportional-integral-derivative controller, whose input is the error between the desired position command and the actual position feedback grating ruler reading, and whose output is used to suppress random disturbances such as environmental vibration or thermal drift. The parameters of the proportional gain, integral gain, and derivative gain are initially tuned based on the model identified by the system and can be fine-tuned during operation according to the motion state. The feedforward control loop is based on an inverse dynamics model, which describes the dynamic characteristics of the piezoelectric ceramic actuator and the nano-positioning platform, including mass, damping, stiffness, and the inherent hysteresis nonlinearity and creep effects of piezoelectric ceramics. The feedforward controller calculates a feedforward compensation force based on the desired motion trajectory command velocity and acceleration using the inverse model, and directly adds it to the control output to counteract the known dynamic characteristics of the system and improve tracking bandwidth. The unique feature of the iterative learning controller is its built-in error memory and learning unit. This unit maintains a database in memory, recording complete information for each historical injection task, including task identifier, target cell mechanical property classification, specific motion trajectory command executed, and trajectory tracking error for each control cycle. When the system begins a new injection task, the learning unit searches the historical database for similar historical task records based on the current target cell's mechanical property parameter vector and the planned motion trajectory characteristics. If sufficiently similar records are found, the learning unit extracts the historical tracking error data from these records during the trajectory similarity phase. Then, it uses a weighted average learning algorithm to generate an additional feedforward compensation signal based on the historical error. This compensation signal is injected into the control command in advance during the next similar motion phase. For example, if historical data shows that the system always has a hysteresis error of about 5 nanometers in the Z-axis direction when performing rapid puncture on a certain type of highly viscoelastic cell, then the learning unit will add a 5-nanometer compensation amount to the Z-axis command in advance when performing a similar action. As the number of similar tasks accumulates, the error data collected by the learning unit becomes richer, and the compensation signal it generates becomes more and more accurate, thereby achieving a gradual elimination of the system's repeatability error, so that the trajectory tracking accuracy gradually improves from the initial submicron level and stabilizes at the nanometer level.

[0041] The fifth core module is the abnormal state self-diagnosis and recovery module. For example... Figure 5As shown, this module acts as the system's safety guardian, continuously monitoring the entire system's operational status to quickly identify and autonomously handle various faults that may occur during operation. The module's anomaly detection function is based on a set of dedicated pattern recognition classifiers running in parallel. These classifiers are also deployed on the graphics processor of the intelligent decision layer to ensure real-time performance. For the common anomaly of needle tip clogging, a clogging classifier based on support vector machines and statistical feature extraction is deployed. This classifier analyzes signals from the force sensor in real time. During normal puncture, when the needle tip penetrates the cell membrane, the force signal exhibits a sharp peak followed by a rapid decline. When partial or complete clogging occurs at the needle tip, the characteristics of the force signal change significantly. The clogging classifier calculates three key features in real time. The first feature is the steady-state resistance value, i.e., the average value of the force signal during continuous feeding. The second feature is the vibrational energy of the high-frequency components of the force signal in the 100 Hz to 5000 Hz frequency band. During normal puncture, the interaction between the needle tip and the cell membrane generates specific high-frequency vibrations, which are suppressed by clogging. The third feature is the slope of the rising edge of the force signal. The classifier compares these three features in real time with a baseline model of a "clean tip" state established through extensive experiments. When steady-state resistance consistently exceeds a threshold (e.g., 200 nanonewtons), high-frequency vibration energy drops below 30% of the baseline level, and the force rise slope is abnormally gentle, the classifier determines a tip blockage anomaly with over 95% confidence and generates an anomaly event message containing an anomaly type code and severity level. For unexpected cell displacement anomalies, a displacement classifier based on computer vision and optical flow is deployed. This classifier receives the sub-pixel coordinate sequence of the cell centroid from a multimodal perception fusion module in real time. It first predicts and tracks the centroid motion using a Kalman filter. Simultaneously, it extracts multiple high-contrast feature points on the culture medium substrate from the microscopic image, such as scratches or impurities on the bottom of the culture dish, and calculates the optical flow field of these feature points to compensate for possible movement of the culture dish itself. The displacement classifier calculates two key metrics in real time. The first is the displacement of the cell centroid relative to the motion-compensated background. The second parameter is the cell contour deformation parameter, which is evaluated by calculating the Hausdorff distance change of the cell contour between consecutive frames. When the displacement of the cell centroid in any direction exceeds a preset safety threshold, such as 2 micrometers, or when the cell contour undergoes unexpected and rapid deformation, the displacement classifier determines that an abnormal cell displacement has occurred and marks the possible causes of the abnormality, such as fluid flow or cell adhesion failure. Once this module confirms any abnormal event, it immediately sends an abnormal event message to the adaptive motion planning and decision-making center through a highest-priority interrupt signal. Upon receiving the message, the decision-making center immediately suspends the current regular motion planning loop. Based on the abnormality type encoding, the decision-making center adopts one of two recovery strategies. For well-defined abnormalities such as needle tip blockage, the decision-making center calls the corresponding recovery protocol from the built-in preset strategy library.For example, in the case of blockage, the protocol might include immediately stopping the feed, instructing the feedforward-feedback composite precision execution module to retract the injection needle 50 micrometers along its original path, and then controlling an auxiliary piezoelectric micro-cleaning device to perform high-frequency vibration cleaning of the needle tip. After cleaning, the needle tip is re-visually positioned, and a new safe path is planned to approach the cell. For more complex or undefined anomalies, the decision center, based on the complete environmental state perceived at the current moment, including the new cell position, needle tip position, force sensor readings, etc., utilizes its reinforcement learning framework's planning capabilities to plan an optimal retraction trajectory in real time to ensure safety, moving the needle tip away from the cell to avoid collision damage. Subsequently, a repositioning and re-approach trajectory is planned. After the recovery strategy is generated, the decision center sends a new sequence of motion instructions to the feedforward-feedback composite precision execution module for execution. After the anomaly is resolved, it reports recovery completion to the task management layer. The system then decides, based on task logic, whether to continue the original task or move to the next work unit.

[0042] The above specific embodiments are merely several preferred embodiments of the present invention. Based on the technical solutions of the present invention and the relevant teachings of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.

Claims

1. An artificial intelligence-based cell injection instrument motion intelligent control system, characterized in that: The system comprises a multi-modal perception fusion module, an online cell mechanics property identification module, an adaptive motion planning and decision-making hub, a feedforward and feedback compound execution module, and an abnormal state autonomous diagnosis and recovery module. The multi-modal perception fusion module collects and synchronously processes microscopic visual data, force sensing data, displacement data, and environmental state data from the operating environment of the cell injector in real time, and generates a fusion perception feature tensor under a unified space-time reference. The online cell mechanics property identification module receives the fusion perception feature tensor from the multi-modal perception fusion module, and through an embedded deep neural network model, calculates the equivalent elastic modulus, membrane tension threshold, and viscous damping coefficient of the current target cell in real time, and outputs a dynamic cell mechanics property parameter vector. The adaptive motion planning and decision-making hub receives the cell mechanics property parameter vector output by the online cell mechanics property identification module, and based on a preset injection task target, generates a real-time and optimized injection needle three-dimensional motion trajectory sequence, axial feed speed curve, and puncture force threshold curve based on a reinforcement learning framework. The reinforcement learning framework used by the adaptive motion planning and decision-making hub is a proximal policy optimization reinforcement learning framework. The cell injection process is modeled as a Markov decision process by the proximal policy optimization reinforcement learning framework. The state space of the Markov decision process is defined as the cell mechanics property parameter vector at the current time, the six-degree-of-freedom pose of the injection needle tip relative to the target cell, and the historical motion trajectory segment. The action space of the Markov decision process is defined as the displacement increment of the injection needle in the three-dimensional space, the axial feed speed instruction, and the maximum allowed contact force instruction in the next control period. The reward function of the Markov decision process is composed of the weighted sum of multiple sub-rewards, including a successful puncture reward, a cell morphology integrity preservation reward, an operation time consumption penalty, and a motion smoothness reward. The feedforward and feedback compound execution module receives the motion instructions issued by the adaptive motion planning and decision-making hub, drives the piezoelectric ceramic actuator and nanometer positioning platform to execute the motion, and at the same time, through an embedded iterative learning controller, compensates and learns the trajectory tracking error in the execution process in real time, to ensure the consistency of the actual motion and the instructed motion. The abnormal state autonomous diagnosis and recovery module continuously monitors the raw data stream of the multi-modal perception fusion module and the state feedback of the feedforward and feedback compound execution module, detects needle tip blockage, cell unexpected displacement, and culture medium interface disturbance abnormal events in real time through a pattern recognition algorithm, and triggers a predefined or dynamically generated recovery strategy by the decision-making hub. 2.The motion intelligent control system of an artificial intelligence-based cell injector according to claim 1, wherein: The multi-modal perception fusion module comprises a digital microscope camera, a nanoscale resolution capacitive force sensor, a grating ruler displacement encoder, and a temperature and humidity sensor; a feature extraction and synchronization unit is arranged inside the multi-modal perception fusion module; the feature extraction and synchronization unit performs real-time cell edge and needle tip sub-pixel positioning on the microscopic image sequence, denoising and feature extraction on the force sensor signal, and differential calculation on the displacement encoder signal to obtain real-time speed and acceleration, and aligns the time stamps of all data streams to the system master clock; the output of the feature extraction and synchronization unit is a four-dimensional tensor, and the dimensions correspond to time sequence, spatial position, sensor type and feature channel. 3.The motion intelligent control system of an artificial intelligence-based cell injector according to claim 1, wherein: The deep neural network model built in the cell mechanics property online identification module is a cascaded deep neural network model; the first stage of the cascaded deep neural network model is a convolutional neural network branch, which processes microscopic image features from the multi-modal perception fusion module to predict the morphological category and apparent hardness of the cell; the second stage of the cascaded deep neural network model is a time series recurrent neural network branch, which processes microforce and displacement time series feature pairs from the multi-modal perception fusion module to fit the force and displacement response curve of the cell in the initial stage of the micro-needle contact; the third stage of the cascaded deep neural network model is a fully connected fusion network, which inputs the output feature vectors of the first and second stages of the network, and directly regresses the cell mechanics property parameter vector containing three scalar parameters after multi-layer nonlinear transformation. 4.The motion intelligent control system of an artificial intelligence-based cell injector according to claim 1, wherein: The adaptive motion planning and decision center comprises a strategy network and a value network; the strategy network outputs a probability distribution of actions according to the current state; the value network evaluates the long-term cumulative reward expectation of the current state; during the execution of each injection task, the state-action-reward sequence collected by the system is used to adjust the parameters of the strategy network and the value network online. 5.The motion intelligent control system of an artificial intelligence-based cell injector according to claim 1, wherein: The feedforward and feedback compound execution module comprises a piezoelectric ceramic actuator, a nanometer positioning platform, a digital-to-analog conversion and power amplification circuit, and an iterative learning controller; the iterative learning controller takes feedforward and feedback compound control as the core, the feedback link adopts a proportional-integral-derivative controller to suppress random disturbances; the feedforward link adopts model-based inverse dynamics compensation to offset the known nonlinear and lag characteristics of the system. 6.The motion intelligent control system of an artificial intelligence-based cell injector according to claim 5, characterized in that: An error memory and learning unit is built in the iterative learning controller; The error memory and learning unit records the history of trajectory tracking errors when the same type of cells is executed in the motion trajectory in the current injection task; in the subsequent motion stage, the error memory and learning unit generates a feedforward compensation signal based on the historical error data, which is directly superimposed into the control command. 7.The motion intelligent control system of an artificial intelligence-based cell injector according to claim 1, wherein: The abnormality detection of the abnormality self-diagnosis and recovery module is based on a set of parallel special classifiers; for the needle tip clogging abnormality, the classifier analyzes the force sensor signal and the steady-state resistance value in real time and compares it with the historical baseline of cleaning the needle tip; when the resistance continuously exceeds the threshold value and the vibration energy decreases, it is determined that the needle tip is clogged; for the cell unexpected displacement abnormality, the classifier compares the center of mass positions of the target cells in the consecutive frame microscopic images, and combines the feature points of the culture substrate to calculate the motion compensation; when it is detected that the cells have rigid displacement or deformation beyond the preset range, it is determined that the displacement is abnormal. 8.The motion intelligent control system of an artificial intelligence-based cell injector according to claim 7, characterized in that: Once the abnormality self-diagnosis and recovery module detects a confirmed abnormal event, it immediately sends an interrupt signal and an abnormal type code to the adaptive motion planning and decision center; The adaptive motion planning and decision center then calls the corresponding recovery protocol from the preset strategy library according to the abnormal type, or plans a new safe retreat trajectory and repositioning trajectory based on the current environment state in real time, and instructs the feedforward and feedback composite execution module to execute. 9.The motion intelligent control system of an artificial intelligence-based cell injector according to claim 1, wherein: The system runs in a hierarchical real-time control architecture; the hierarchical real-time control architecture includes a task management layer, an intelligent decision layer and a real-time control layer; the task management layer runs at a time scale of seconds, is responsible for task analysis, batch planning and human-computer interaction; the intelligent decision layer runs at a time scale of milliseconds, includes the functions of the adaptive motion planning and decision center and the abnormality self-diagnosis and recovery module, and makes online decisions and plans; the real-time control layer runs at a time scale of microseconds, includes the bottom control loop of the feedforward and feedback composite execution module, and ensures the execution of the instructions; The three layers exchange data through a deterministic real-time communication bus.

Images