Cell injection instrument motion intelligent control system based on artificial intelligence

Abstract

The invention relates to the technical field of motion control of cell injectors, in particular to an intelligent control system for motion of a cell injector based on artificial intelligence, and adopts the technical scheme that a multi-modal sensing fusion module and a cell mechanical property online identification module are constructed; the problems of poor universality and high injury risk caused by diversity of biological samples are fundamentally solved; a self-adaptive motion planning and decision center based on reinforcement learning is introduced, and a motion control problem is converted into a continuous learning optimization process; a designed feedforward and feedback composite execution module, especially a built-in iterative learning controller, effectively overcomes the bottleneck that repetitive errors in a precise motion system are difficult to eliminate; an integrated abnormal state autonomous diagnosis and recovery module endows the system with high robustness and fault-tolerant capability; through parallelized real-time mode recognition, the system can quickly and accurately diagnose various common operation faults and trigger an intelligent recovery strategy, so that the dependence on manual intervention is greatly reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cell injector motion control, and particularly relates to an intelligent motion control system for a cell injector based on artificial intelligence. BACKGROUND

[0002] In the field of precision instruments and automation control, one of the core technical challenges is to achieve precise motion control at the micron or even nanometer level, which is widely used in biomedical, microelectronic manufacturing, and material science, etc. Among them, cell injection as a key biological operation technology, its core goal is to accurately, efficiently and non-destructively inject exogenous substances (such as genes, drugs or tracers) into target cells through high-precision mechanical motion.

[0003] Among them, the automatic motion control of the cell injector is a key technology direction to improve the injection success rate and operation efficiency. This technology aims to replace the traditional manual injection mode relying on manual operation of the microscope by presetting or adjusting the motion trajectory and parameters in real time to drive the injection needle to complete a series of complex actions such as positioning, puncturing and injection of cells, so as to overcome the instability of the results and the low throughput bottleneck caused by the skill difference of the operator.

[0004] The existing technology usually adopts a rigid motion control strategy based on a preset program, or combines simple visual feedback for position correction. However, such methods have significant limitations: first, the control model is difficult to adapt to the inherent diversity and dynamics of biological samples, such as changes in mechanical properties caused by different cell types, sizes, shapes and culture medium differences, making fixed puncture force or speed parameters prone to cause cell damage or injection failure. Secondly, the existing system lacks deep fusion analysis and intelligent reasoning ability for the multi-modal data (such as micro-image sequences, pressure sensing signals, displacement feedback) generated in real time during the injection process, and cannot learn from the data and dynamically optimize the control strategy. Thirdly, in the face of high-throughput injection demand of cell groups, the existing control logic shows insufficient self-adaptation in task planning, path optimization and abnormal handling (such as needle tip blockage, cell drift), and relies heavily on manual intervention, which restricts the automation level and reliability of the overall operation. Therefore, how to build an intelligent control system that can perceive complex operating environment, make autonomous decisions and optimize motion behavior in real time has become a core problem to be solved to improve the efficiency of cell injection technology. SUMMARY

[0005] The present application aims to provide an intelligent motion control system for a cell injector based on artificial intelligence to solve the problems of high cell damage rate, unstable injection success rate and limited automation level caused by the rigid control model, insufficient multi-modal data fusion analysis ability and lack of self-adaptation to the diversity and dynamics of biological samples in the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: The AI-based intelligent motion control system for cell injection devices includes a multimodal perception fusion module, an online cell mechanical property identification module, an adaptive motion planning and decision-making center, a feedforward and feedback composite execution module, and an abnormal state autonomous diagnosis and recovery module. Among them, the multimodal perception fusion module collects and processes microscopic visual data, force sensing data, displacement data and environmental state data from the cell injection instrument operating environment in real time and synchronously, and generates a fusion perception feature tensor under a unified spatiotemporal reference. The online cell mechanical property identification module receives the fused sensing feature tensor from the multimodal sensing fusion module. Through the built-in deep neural network model, it calculates the equivalent elastic modulus, membrane tension threshold, and viscous damping coefficient of the current target cell in real time and outputs a dynamic vector of cell mechanical property parameters. The adaptive motion planning and decision-making center receives the cell mechanical property parameter vector output by the cell mechanical property online identification module, and combines it with the preset injection task objective to generate a real-time, optimized three-dimensional motion trajectory sequence of the injection needle, axial feed velocity curve, and puncture force threshold curve based on the reinforcement learning framework. The feedforward and feedback composite execution module receives motion commands from the adaptive motion planning and decision-making center, drives the piezoelectric ceramic actuator and nano-positioning platform to perform motion, and at the same time, through the built-in iterative learning controller, performs online compensation and learning for trajectory tracking errors during the execution process to ensure the consistency between actual motion and commanded motion. The abnormal state autonomous diagnosis and recovery module continuously monitors the raw data stream of the multimodal perception fusion module and the state feedback of the feedforward and feedback composite execution module. It uses pattern recognition algorithms to detect abnormal events such as needle tip blockage, unexpected cell displacement, and culture medium interface disturbance in real time, and triggers predefined or dynamically generated recovery strategies by the decision center.

[0007] Furthermore, the multimodal perception fusion module includes a digital microscope camera, a nanometer-resolution capacitive force sensor, a grating ruler displacement encoder, and a temperature and humidity sensor. Internally, the multimodal perception fusion module deploys a feature extraction and synchronization unit. This unit performs real-time cell edge and pinpoint sub-pixel localization on the microscopic image sequence, performs noise reduction and feature extraction on the force sensor signal, performs differential calculation on the displacement encoder signal to obtain real-time velocity and acceleration, and aligns the timestamps of all data streams to the system master clock. The output of the feature extraction and synchronization unit is a four-dimensional tensor, with dimensions corresponding to the time series, spatial location, sensor type, and feature channel, respectively.

[0008] Further, the deep neural network model built-in the online cell mechanical property identification module is a cascaded deep neural network model; the first stage of the cascaded deep neural network model is a convolutional neural network branch, which processes microscopic image features from the multi-modal perception fusion module to predict the morphological category and apparent stiffness of the cell; the second stage of the cascaded deep neural network model is a time-series recurrent neural network branch, which processes pairs of micro-force and displacement time-series features from the multi-modal perception fusion module to fit the force and displacement response curve of the cell in the initial stage of the microneedle contact; the third stage of the cascaded deep neural network model is a fully connected fusion network, which takes the output feature vectors of the first and second stages as inputs, and directly regresses a cell mechanical property parameter vector containing three scalar parameters after multi-layer nonlinear transformation.

[0009] Further, the reinforcement learning framework based on which the adaptive motion planning and decision-making hub is a proximal policy optimization reinforcement learning framework; the cell injection process is modeled as a Markov decision process by the proximal policy optimization reinforcement learning framework; the state space of the Markov decision process is defined as the cell mechanical property parameter vector at the current time, the six-degree-of-freedom pose of the injection needle tip relative to the target cell, and the historical motion trajectory segment; the action space of the Markov decision process is defined as the displacement increment of the injection needle in three-dimensional space, the axial feed rate command, and the maximum allowed contact force command in the next control period; the reward function of the Markov decision process is composed of the weighted sum of multiple sub-rewards, including the successful puncture reward, the cell morphology integrity preservation reward, the operation time consumption penalty, and the motion smoothness reward.

[0010] Further, the adaptive motion planning and decision-making hub contains a policy network and a value network; the policy network outputs the probability distribution of actions according to the current state; the value network evaluates the long-term cumulative reward expectation of the current state; during the execution of each injection task, the state-action-reward sequence collected by the system is used to adjust the parameters of the policy network and the value network online.

[0011] Further, the feedforward and feedback compound execution module contains a piezoelectric ceramic actuator, a nanometer positioning platform, a digital-to-analog conversion and power amplification circuit, and an iterative learning controller; the iterative learning controller takes feedforward and feedback compound control as the core, with a proportional-integral-derivative controller in the feedback loop to suppress random disturbances; and a model-based inverse dynamics compensation in the feedforward loop to offset the known nonlinear and lag characteristics of the system.

[0012] Further, an error memory and learning unit is built-in the iterative learning controller; the error memory and learning unit records the history of trajectory tracking errors when executing the same type of motion trajectory for cells in the current injection task; in the subsequent motion stage, the error memory and learning unit generates a feedforward compensation signal based on the historical error data, which is directly superimposed into the control command.

[0013] Further, the abnormal state autonomous diagnosis and recovery module detects abnormalities based on a set of parallel dedicated classifiers; for needle clogging abnormality, the classifier analyzes force sensor signals and steady-state resistance values in real time and compares them with historical baseline of cleaning needle tip, and when resistance continues to exceed threshold and vibration energy decreases, it is determined that clogging occurs; for cell unexpected displacement abnormality, the classifier compares the center of mass position of the target cell in consecutive frame microscopic images, and calculates motion compensation combined with the feature points of the culture substrate, and when it is detected that the cell has rigid displacement or deformation beyond the preset range, it is determined that displacement abnormality occurs.

[0014] Further, the abnormal state autonomous diagnosis and recovery module sends an interrupt signal and an abnormal type code to the adaptive motion planning and decision center as soon as a confirmed abnormal event is detected; the adaptive motion planning and decision center calls the corresponding recovery protocol from the preset strategy library according to the abnormal type, or plans a new safe retreat trajectory and repositioning trajectory in real time based on the current environment state, and instructs the feedforward and feedback composite execution module to execute.

[0015] Further, the system operates in a hierarchical real-time control architecture; the hierarchical real-time control architecture includes a task management layer, an intelligent decision layer and a real-time control layer; the task management layer operates at a time scale of seconds, is responsible for task analysis, batch planning and human-computer interaction; the intelligent decision layer operates at a time scale of milliseconds, includes the functions of the adaptive motion planning and decision center and the abnormal state autonomous diagnosis and recovery module, and makes online decision and planning; the real-time control layer operates at a time scale of microseconds, includes the bottom control loop of the feedforward and feedback composite execution module, and ensures the execution of instructions; the three layers exchange data through a deterministic real-time communication bus.

[0016] Compared with the prior art, the present application has the following advantages: 1、The present application realizes real-time quantitative perception and calculation of individual and dynamic mechanical properties of living cells by constructing a multi-modal perception fusion module and an online cell mechanics identification module; the system discards the fixed parameter model, can adaptively adjust the control strategy according to the real-time response of each cell, fundamentally solves the problems of poor universality and high damage risk caused by the diversity of biological samples, and improves the cell injection operation from experience-based "blind operation" to data-based "perception operation".

[0017] 2、The application introduces an adaptive motion planning and decision-making center based on reinforcement learning, which converts the motion control problem into an optimization process of continuous learning; the system can autonomously learn the optimal injection strategy through interaction with the environment in actual operation, not only optimizing the success rate of single injection and cell survival rate, but also achieving comprehensive optimization of operation speed, smoothness and other multi-objectives through the guidance of reward function; this combination of data-driven and model-driven methods significantly improves the intelligence level and autonomous decision-making ability of the system in the face of complex and unstructured operating environment.

[0018] 3、The designed feedforward and feedback composite execution module, especially the built-in iterative learning controller, effectively overcomes the bottleneck of repetitive error that is difficult to eliminate in precision motion systems; the controller can actively learn and compensate for the inherent nonlinearities, hysteresis and other characteristics of the system using historical execution data, so that the motion control precision can evolve and improve with the accumulation of operation times, providing reliable technical support for achieving and long-term stable maintaining sub-micron or even nanometer level positioning and tracking precision.

[0019] 4、The integrated abnormal state autonomous diagnosis and recovery module gives the system high robustness and fault tolerance; through parallel real-time pattern recognition, the system can quickly and accurately diagnose various common operation faults and trigger intelligent recovery strategies, greatly reducing the dependence on manual intervention; this not only improves the reliability and efficiency of high-throughput continuous operation, but also reduces the risk of sample loss caused by operation interruption or failure, achieving fully automatic and unattended cell injection operation. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 is the overall technical scheme architecture schematic diagram of the motion intelligent control system of the cell injector based on artificial intelligence proposed by the application; Figure 2 is the core principle framework schematic diagram of the online identification module of cell mechanics characteristics in the application; Figure 3 is the logic flow framework diagram of the adaptive motion planning and decision-making center in the application; Figure 4 is the multi-level interaction relationship and data flow schematic diagram of the feedforward and feedback composite execution module in the application; Figure 5 is the interaction and recovery strategy triggering schematic diagram of the abnormal state autonomous diagnosis and recovery module and other parts of the system in the application. DETAILED DESCRIPTION

[0021] The technical scheme of the application is further described below in conjunction with the drawings and specific embodiments.

[0022] Example 1:

[0023] This invention provides an artificial intelligence-based intelligent control system for a cell injection device. For example... Figure 1 As shown, the system physically implements a hierarchical real-time control architecture, comprising a task management layer, an intelligent decision-making layer, and a real-time control layer from top to bottom. The task management layer operates on a second-level timescale, and its core function is to parse the injection task parameters input by the user through the human-computer interaction interface. These parameters include, but are not limited to, target cell type, expected injection site, injection substance volume, and the number and arrangement of cells in the batch. Based on these parameters, the task management layer generates macroscopic task sequences and logical flows, and issues task instructions and status query requests to the lower layers via a deterministic real-time communication bus. The intelligent decision-making layer operates on a millisecond-level timescale and is the core carrier of the system's intelligence. It receives macroscopic instructions from the task management layer and processes data streams from the perception layer in real time, performing online decision-making, planning, and anomaly diagnosis. The real-time control layer operates on a microsecond-level timescale, directly driving the physical actuators and ensuring that the motion instructions issued by the intelligent decision-making layer are executed with high precision and high bandwidth. The three layers exchange data via a time-triggered real-time Ethernet bus, with the bus cycle strictly synchronized to the system's master clock, ensuring deterministic low latency and high reliability in the transmission of status information and instructions.

[0024] The perception and decision-making functions of the system are realized collaboratively by five core modules. The first core module is the multi-modal perception fusion module. As the front-end of the system's perception, this module is responsible for real-time acquisition and synchronous processing of multi-source heterogeneous sensor data from the cell injection instrument operating environment. Its hardware integration includes a high-speed digital microscope camera with a frame rate of no less than 2000 frames per second, equipped with a high numerical aperture objective lens and a long working distance condenser, which can provide sub-micron spatial resolution bright-field or phase-contrast microscopic image sequences. A nanometer-resolution capacitive micro-force sensor with a measurement range of plus or minus 100 micro-newtons and a resolution better than 10 nanonewtons, which is directly integrated into the holder of the injection needle, is used to measure the three-dimensional contact force during the contact of the needle tip with the cell and the culture medium. A grating ruler displacement encoder system, which includes three orthogonal direction grating rulers and reading heads, with a measurement resolution of 0.1 nanometers, is used to feedback the absolute position of the injection needle driving platform in three-dimensional space. And a set of environmental state sensors, including temperature and humidity sensors and carbon dioxide concentration sensors integrated near the culture dish, are used to monitor the stability of the cell culture environment. A dedicated feature extraction and synchronization unit is deployed inside the multi-modal perception fusion module, which is realized by a field programmable gate array and a digital signal processor. For microscopic image sequences, the feature extraction and synchronization unit performs a real-time image processing pipeline, first performing Gaussian filter denoising, then using the Canny edge detection algorithm combined with sub-pixel interpolation technology to accurately extract the contour edge of the target cell and the sub-pixel coordinates of the injection needle tip, calculate the centroid position, equivalent diameter, roundness of the cell, and the distance and angle of the needle tip relative to the centroid of the cell. For micro-force sensor signals, the unit first performs hardware synchronous acquisition with a sampling rate of 100 kilohertz per second, then filters out high-frequency electrical noise through a fourth-order Butterworth low-pass filter, and calculates the force signal components in three axes, the total force, the change gradient, and the vibration energy spectrum in a specific frequency band such as 100 hertz to 1000 hertz in real time. For grating ruler displacement encoder signals, the unit performs high-speed differential calculation to obtain the real-time speed and acceleration of the injection needle in three directions. All data streams are marked with accurate time stamps from the same system master clock before entering the processing pipeline, with a time stamp accuracy of 1 microsecond. The feature extraction and synchronization unit finally aligns and packages all processed features according to a unified space-time reference, outputting a structured four-dimensional fusion perception feature tensor. The first dimension of this tensor is the time series index, corresponding to the continuous sampling time. The second dimension is the spatial grid index, which divides the field of view into a 32x32 grid, and each grid stores the features of that region. The third dimension is the sensor type index, corresponding to visual features, force perception features, displacement features, and environmental features. The fourth dimension is the feature channel, for example, for the visual feature channel, it may include edge intensity, grayscale gradient, texture features, and a total of 128 channel scalar values.This four-dimensional tensor is pushed to downstream modules in real time via a high-speed data bus.

[0025] The second core module is the online identification module for cell mechanical properties. For example... Figure 2As shown, the module receives the fused perception feature tensor from the multi-modal perception fusion module, and its core task is to solve the individualized and dynamic mechanical property parameters of the current target cell in real time. The core of the module is a pre-trained cascaded deep neural network model, which is deployed on a dedicated graphics processor in the intelligent decision-making layer to ensure millisecond-level inference speed. The model is composed of three cascaded sub-networks. The first level is a convolutional neural network branch, whose input is the feature slice related to vision in the fused perception feature tensor. The branch contains 5 convolutional layers and 3 pooling layers, and the convolution kernel size decreases from 7x7 to 3x3 layer by layer, and the channel number increases from 64 to 256. This branch is specifically used to extract high-level morphological and texture features from the microscopic image of the cell, and output a 256-dimensional feature vector that encodes the morphological category information and apparent hardness prior information of the cell, such as whether the cell belongs to fibroblast cells, neuron cells or oocytes, and whether its membrane surface is wrinkled or smooth. The second level is a time series recurrent neural network branch, specifically using a long short-term memory network structure. Its input is the feature sequence related to the micro-force-displacement time series in the fused perception feature tensor, and the sequence length is usually 100 sampling time data in the recent 100 sampling time data. The long short-term memory network contains 2 hidden layers, each with 128 units. It is specifically used to dynamically model the dynamic response relationship between the force and displacement exhibited by the cell at the initial stage of the needle contact, and learn its creep, relaxation and other viscoelastic behaviors. The branch outputs a 128-dimensional feature vector that encodes the dynamic mechanical response mode of the cell. The third level is a fully connected fusion network, whose input is the 256-dimensional feature vector output by the first level convolutional neural network branch and the 128-dimensional feature vector output by the second level long short-term memory network branch, which are spliced into a 384-dimensional vector. The fully connected network contains 3 hidden layers with 256, 128, and 64 neurons respectively, and each hidden layer is followed by a batch normalization layer and a ReLU activation function. The final output layer of the network is a linear layer that directly outputs a cell mechanical property parameter vector containing 3 scalar parameters. The 3 parameters have clear physical meaning and units. The first parameter is the equivalent elastic modulus of the cell, with a unit of kilopascal, which represents the cell's resistance to elastic deformation. The second parameter is the membrane tension threshold of the cell, with a unit of nanonewton per micrometer, which represents the maximum tension that the cell membrane can withstand before breaking. The third parameter is the viscous damping coefficient of the cell, with a unit of nanonewton second per micrometer, which represents the energy dissipation characteristics caused by the internal cytoplasm and membrane flow of the cell. Before system deployment, the deep neural network model must be fully trained offline. The training data set is composed of a large number of experiments, including standard polyacrylamide microspheres with known mechanical parameters, and various types of real cells such as HeLa cells, MCF-7 cells, and mouse oocytes. The accurate mechanical parameter true values of these samples are obtained through standard mechanical testing methods such as microtube suction and atomic force microscope nanoindentation.Simultaneously, multimodal sensing data of these samples were collected under simulated injection conditions, including images, force curves, and displacement curves, forming an input-output paired dataset. Training employed a mean squared error loss function and the Adam optimizer, undergoing over 1 million iterations to enable the model to accurately regress the cell's mechanical properties from the multimodal sensing data. During online operation, the module received the latest fused sensing feature tensor at a frequency of 1000 times per second and performed forward inference, outputting a dynamically updated vector of cell mechanical property parameters, providing crucial information for motion planning.

[0026] The third core module is the adaptive motion planning and decision-making center. For example... Figure 3 As shown, this module is the core decision engine of the intelligent decision layer. It receives the cell mechanical property parameter vector output in real time from the online cell mechanical property identification module and, combined with the injection task objective issued by the task management layer, generates the optimal injection needle motion command. Its operating mechanism is based on a proximal policy optimization reinforcement learning framework, formalizing the entire cell injection process as a Markov decision process. The mathematical model of this process is as follows: The system's state space S is a high-dimensional continuous vector. At each decision time t, the state st is specifically defined as the concatenation of three parts of information. The first part is the cell mechanical property parameter vector at the current time, namely the equivalent elastic modulus Et, membrane tension threshold Tt, and viscous damping coefficient Ct. The second part is the six-degree-of-freedom pose of the injection needle tip relative to the target cell, including three position coordinates xt, yt, zt and three Euler angle poses θxt, θyt, θzt. This information is calculated by fusing visual positioning and displacement encoder data provided by the multimodal perception fusion module. The third part is the historical motion trajectory segment, namely the velocity and acceleration sequence of the injection needle in the past 10 decision cycles. The system's action space A is also a continuous space. At each decision time t, the action at output by the agent, i.e., the decision center, is a 7-dimensional vector. The first three dimensions define the displacement increments Δx, Δy, and Δz of the injection needle in the three-dimensional task space within the next control cycle, in micrometers. The fourth dimension defines the feed velocity command vz of the injection needle along its axis, in micrometers per second. Dimensions 5 to 7 define the maximum permissible contact force commands Fmaxx, Fmaxy, and Fmaxz in the three coordinate axes, in nanonewtons, used to protect cells from excessive lateral force damage. The system's reward function R is key to driving the agent's learning; it consists of a weighted sum of multiple carefully designed sub-rewards. Specifically, the reward function is defined as: R(t)=w1*R(puncture)+w2*R(integrity)+w3*R(time)+w4*R(smoothness) where R(puncture) is the success puncture reward, a large positive reward is given when the needle tip detects a breakthrough of the cell membrane and the micro-force sensor shows a sudden drop in force value. R(integrity) is the cell morphology integrity preservation reward, by calculating the roundness change rate of the cell contour in real time, if the change rate is lower than the threshold, a continuous positive reward is given, if the cell deforms severely, a negative reward is given. R(time) is the operation time consumption penalty, a small negative reward is given in each decision cycle to encourage fast operation. R(smoothness) is the motion smoothness reward, by calculating the second order norm of the change of motion in adjacent cycles, the smaller the change is, the higher the reward is, to encourage the generation of smooth motion trajectory. The weight coefficients w1, w2, w3, w4 are determined by offline simulation debugging, for example, set to 10, 2, -0.01, 0.5 respectively. The adaptive motion planning and decision-making center contains two neural networks, the policy network πθ and the value network Vφ. The policy network πθ takes the current state st as input, processes it through 3 fully connected layers with 256 neurons in each layer, and outputs the mean μ and variance σ of each dimension in the action at, thereby defining a Gaussian probability distribution, from which the agent samples the final executed action. The value network Vφ also takes the state st as input, and evaluates the expected value of the cumulative discounted reward that can be obtained in the future. When the system is initially deployed, the policy network and the value network have been pre-trained for millions of rounds in a high-fidelity physical simulation environment, and have mastered the basic injection skills. In actual operation, every time an injection task or a decision segment is completed, the system will collect a series of state, action, reward sequences. These sequences are sent to the proximal policy optimization algorithm for online fine-tuning. The algorithm calculates the advantage function and applies a constraint on the magnitude of policy updates to safely update the parameters θ and φ of the policy network πθ and the value network Vφ. This allows the system's motion planning strategy to continuously adapt and optimize based on the diversity of cells encountered in actual operation, environmental disturbances, and other factors, constantly approaching the optimal control strategy for the current specific sample.

[0027] The fourth core module is the feedforward and feedback composite execution module. As shown in FIG. 6, the feedforward and feedback composite execution module is composed of a feedforward motion planning and decision-making center, a motion controller, a force sensor, a needle tip position sensor, a cell image sensor, a cell image processing module, a feedback control module, and a needle tip position control module. Figure 4As shown, this module belongs to the real-time control layer and is responsible for converting abstract motion commands issued by the adaptive motion planning and decision-making center into high-precision physical motion. Its hardware actuators include a set of high-bandwidth piezoelectric ceramic actuators with a stroke range of 100 micrometers and a closed-loop bandwidth greater than 2 kHz. A nano-positioning platform based on a flexible hinge has a motion range of 100 micrometers × 100 micrometers × 100 micrometers in all three degrees of freedom, achieving a resolution of 0.3 nanometers. It also includes a matching high-precision 18-bit digital-to-analog converter and a linear power amplifier to drive the piezoelectric ceramic actuators. The core of this module's control is an iterative learning controller, based on a feedforward and feedback composite control architecture. The feedback control uses a digital proportional-integral-derivative controller, whose input is the error between the desired position command and the actual position feedback grating ruler reading, and whose output is used to suppress random disturbances such as environmental vibration or thermal drift. The parameters of the proportional gain, integral gain, and derivative gain are initially tuned based on the model identified by the system and can be fine-tuned during operation according to the motion state. The feedforward control loop is based on an inverse dynamics model, which describes the dynamic characteristics of the piezoelectric ceramic actuator and the nano-positioning platform, including mass, damping, stiffness, and the inherent hysteresis nonlinearity and creep effects of piezoelectric ceramics. The feedforward controller calculates a feedforward compensation force based on the desired motion trajectory command velocity and acceleration using the inverse model, and directly adds it to the control output to counteract the known dynamic characteristics of the system and improve tracking bandwidth. The unique feature of the iterative learning controller is its built-in error memory and learning unit. This unit maintains a database in memory, recording complete information for each historical injection task, including task identifier, target cell mechanical property classification, specific motion trajectory command executed, and trajectory tracking error for each control cycle. When the system begins a new injection task, the learning unit searches the historical database for similar historical task records based on the current target cell's mechanical property parameter vector and the planned motion trajectory characteristics. If sufficiently similar records are found, the learning unit extracts the historical tracking error data from these records during the trajectory similarity phase. Then, it uses a weighted average learning algorithm to generate an additional feedforward compensation signal based on the historical error. This compensation signal is injected into the control command in advance during the next similar motion phase. For example, if historical data shows that the system always has a hysteresis error of about 5 nanometers in the Z-axis direction when performing rapid puncture on a certain type of highly viscoelastic cell, then the learning unit will add a 5-nanometer compensation amount to the Z-axis command in advance when performing a similar action. As the number of similar tasks accumulates, the error data collected by the learning unit becomes richer, and the compensation signal it generates becomes more and more accurate, thereby achieving a gradual elimination of the system's repeatability error, so that the trajectory tracking accuracy gradually improves from the initial submicron level and stabilizes at the nanometer level.

[0028] The fifth core module is the abnormal state autonomous diagnosis and recovery module. As Figure 5As shown, this module acts as the security guard of the system, continuously monitoring the operational status of the whole system, aiming to quickly identify and autonomously handle various types of faults that may occur during the operation process. The anomaly detection function of this module is based on a set of parallel running special pattern recognition classifiers, which are also deployed on the GPUs of the intelligent decision-making layer to ensure real-time performance. For the common anomaly of needle tip clogging, a clogging classifier based on support vector machine and statistical feature extraction is deployed. This classifier analyzes the signals from the force sensor in real time. During normal puncture, when the needle tip penetrates the cell membrane, the force signal will present a sharp peak followed by a rapid decline. When partial or complete clogging of the needle tip occurs, the characteristics of the force signal will change significantly. The clogging classifier calculates three key features in real time. The first feature is the steady-state resistance value, which is the average value of the force signal during the continuous feeding stage. The second feature is the vibration energy of the force signal in the 100 Hz to 5000 Hz frequency band. During normal puncture, the interaction between the needle tip and the cell membrane produces specific high-frequency vibrations, and clogging will inhibit this vibration. The third feature is the slope of the rising edge of the force signal. The classifier compares these three features with the pre-established "clean needle tip" state baseline model through a large number of experiments in real time. When it is detected that the steady-state resistance continuously exceeds the threshold value, for example, 200 nanonewtons, while the high-frequency vibration energy drops to less than 30% of the baseline level, and the force rising edge slope is abnormally flat, the classifier determines that the needle tip clogging anomaly has occurred with a confidence of more than 95%, and generates an anomaly event message containing the anomaly type code and severity level. For the cell unexpected displacement anomaly, a displacement classifier based on computer vision and optical flow method is deployed. This classifier receives the cell centroid sub-pixel coordinate sequence provided by the multi-modal perception fusion module in real time. It first predicts and tracks the centroid motion through a Kalman filter. At the same time, it extracts multiple high-contrast feature points on the culture substrate from the microscopic image, such as scratches or impurities on the bottom of the culture dish, and calculates the optical flow field of these feature points to compensate for the possible movement of the culture dish itself. The displacement classifier calculates two key indicators in real time. The first is the displacement of the cell centroid relative to the background after motion compensation. The second is the deformation parameter of the cell contour, which is evaluated by calculating the Hausdorff distance change between consecutive frames. When it is detected that the displacement of the cell centroid in any direction exceeds the pre-set safety threshold, for example, 2 microns, or the cell contour undergoes unexpected and rapid deformation, the displacement classifier determines that the cell displacement anomaly has occurred, and marks the possible causes of the anomaly such as liquid flow, cell adhesion failure, etc. Once this module confirms any anomaly event, it immediately sends an anomaly event message to the adaptive motion planning and decision-making hub through the highest priority interrupt signal. After receiving the message, the decision-making hub immediately suspends the current regular motion planning cycle. According to the anomaly type code, the decision-making hub adopts one of the two recovery strategies. For predefined clear anomalies such as needle tip clogging, the decision-making hub calls the corresponding recovery protocol from the built-in preset strategy library.For example, for a blockage, the protocol can include immediately stopping the feeding, instructing the feed-backward compound precision execution module to withdraw the injection needle 50 microns along the original path, then controlling an attached piezoelectric micro-cleaning device to perform high-frequency vibration cleaning on the needle tip, after cleaning, re-performing visual positioning of the needle tip, and planning a new safe path to approach the cell. For more complex or undefined abnormalities, the decision center plans an optimal withdrawal trajectory based on the current state of the entire environment, including the new location of the cell, the position of the needle tip, the force sensor readings, etc., using the planning capabilities of its reinforcement learning framework, to ensure safety and avoid collision damage, and then plans a repositioning and re-approaching trajectory. After the recovery strategy is generated, the decision center sends the new motion instruction sequence to the feed-backward compound precision execution module for execution, and after the abnormality is resolved, reports to the task management layer that the recovery is complete, and the system decides whether to continue the original task or switch to the next work unit according to the task logic.

[0029] The above specific embodiments are only several preferred embodiments of the present application, and based on the technical solutions of the present application and the related inspirations of the above embodiments, those skilled in the art can make various alternative improvements and combinations on the above specific embodiments.

Claims

1. An intelligent motion control system for a cell injection device based on artificial intelligence, characterized in that: It includes a multimodal perception fusion module, an online cell mechanical property identification module, an adaptive motion planning and decision-making center, a feedforward and feedback composite execution module, and an abnormal state autonomous diagnosis and recovery module; Among them, the multimodal perception fusion module collects and processes microscopic visual data, force sensing data, displacement data and environmental state data from the cell injection instrument operating environment in real time and synchronously, and generates a fusion perception feature tensor under a unified spatiotemporal reference. The online cell mechanical property identification module receives the fused sensing feature tensor from the multimodal sensing fusion module. Through the built-in deep neural network model, it calculates the equivalent elastic modulus, membrane tension threshold, and viscous damping coefficient of the current target cell in real time and outputs a dynamic vector of cell mechanical property parameters. The adaptive motion planning and decision-making center receives the cell mechanical property parameter vector output by the cell mechanical property online identification module, and combines it with the preset injection task objective to generate a real-time, optimized three-dimensional motion trajectory sequence of the injection needle, axial feed velocity curve, and puncture force threshold curve based on the reinforcement learning framework. The feedforward and feedback composite execution module receives motion commands from the adaptive motion planning and decision-making center, drives the piezoelectric ceramic actuator and nano-positioning platform to perform motion, and at the same time, through the built-in iterative learning controller, performs online compensation and learning for trajectory tracking errors during the execution process to ensure the consistency between actual motion and commanded motion. The abnormal state autonomous diagnosis and recovery module continuously monitors the raw data stream of the multimodal perception fusion module and the state feedback of the feedforward and feedback composite execution module. It uses pattern recognition algorithms to detect abnormal events such as needle tip blockage, unexpected cell displacement, and culture medium interface disturbance in real time, and triggers predefined or dynamically generated recovery strategies by the decision center.

2. The intelligent motion control system for a cell injection device based on artificial intelligence according to claim 1, characterized in that: The multimodal sensing fusion module includes a digital microscope camera, a nanometer-resolution capacitive force sensor, a grating ruler displacement encoder, and a temperature and humidity sensor. Internally, the module deploys a feature extraction and synchronization unit. This unit performs real-time cell edge and pinpoint subpixel localization on the microscopic image sequence, performs noise reduction and feature extraction on the force sensor signal, performs differential calculation on the displacement encoder signal to obtain real-time velocity and acceleration, and aligns the timestamps of all data streams to the system's master clock. The output of the feature extraction and synchronization unit is a four-dimensional tensor, with dimensions corresponding to the time series, spatial location, sensor type, and feature channel, respectively.

3. The intelligent motion control system for a cell injection device based on artificial intelligence according to claim 1, characterized in that: The deep neural network model built into the online cell mechanical property identification module is a cascaded deep neural network model. The first level of the cascaded deep neural network model is a convolutional neural network branch, which processes the microscopic image features from the multimodal perception fusion module to predict the cell morphology and apparent stiffness. The second level of the cascaded deep neural network model is a temporal recurrent neural network branch, which processes the micro-force and displacement temporal feature pairs from the multimodal perception fusion module to fit the force and displacement response curves of the cell in the initial stage of microneedle contact. The third level of the cascaded deep neural network model is a fully connected fusion network, whose input is the output feature vector of the first and second level networks. After multiple nonlinear transformations, it directly regresses and outputs a cell mechanical property parameter vector containing three scalar parameters.

4. The intelligent motion control system for a cell injection device based on artificial intelligence according to claim 1, characterized in that: The adaptive motion planning and decision-making center is based on a proximal policy optimization reinforcement learning framework; the proximal policy optimization reinforcement learning framework models the cell injection process as a Markov decision process; The state space of a Markov decision process is defined as the current vector of cell mechanical properties, the six-degree-of-freedom pose of the needle tip relative to the target cell, and the historical motion trajectory fragments. The action space of a Markov decision process is defined as the displacement increment of the needle in three-dimensional space, the axial feed velocity command, and the maximum permissible contact force command in the next control cycle. The reward function of a Markov decision process consists of a weighted sum of multiple sub-rewards, including a successful puncture reward, a cell morphology integrity preservation reward, an operation time penalty, and a motion smoothness reward.

5. The intelligent motion control system for a cell injection device based on artificial intelligence according to claim 4, characterized in that: The adaptive motion planning and decision-making center comprises a policy network and a value network. The policy network outputs the probability distribution of actions based on the current state. The value network evaluates the long-term cumulative reward expectation of the current state. During each injection task execution, the state-action-reward sequence collected by the system is used to adjust the parameters of the policy network and the value network online.

6. The intelligent motion control system for a cell injection device based on artificial intelligence according to claim 1, characterized in that: The feedforward and feedback composite execution module includes a piezoelectric ceramic actuator, a nano-positioning platform, a digital-to-analog converter and power amplifier circuit, and an iterative learning controller. The iterative learning controller is based on the feedforward and feedback composite control. Its feedback loop adopts a proportional-integral-derivative controller to suppress random disturbances. Its feedforward loop adopts model-based inverse dynamics compensation to offset the known nonlinearity and hysteresis characteristics of the system.

7. The intelligent motion control system for a cell injection device based on artificial intelligence according to claim 6, characterized in that: The iterative learning controller has a built-in error memory and learning unit; The error memory and learning unit records the history of trajectory tracking errors when performing motion trajectories for the same type of cells during the current injection task; in the subsequent motion phase, the error memory and learning unit generates a feedforward compensation signal based on the historical error data, which is directly superimposed on the control command.

8. The intelligent motion control system for a cell injection device based on artificial intelligence according to claim 1, characterized in that: The abnormality detection of the autonomous diagnosis and recovery module is based on a set of parallel dedicated classifiers. For needle tip blockage, the classifier analyzes the force sensor signal and steady-state resistance value in real time and compares it with the historical baseline of the clean needle tip. When the resistance continues to exceed the threshold and the vibration energy decreases, it is determined to be a blockage. For unexpected cell displacement, the classifier compares the centroid position of the target cell in consecutive frame microscopic images and performs motion compensation calculations by combining the feature points of the culture medium substrate. When rigid displacement or deformation of the cell exceeding the preset range is detected, it is determined to be a displacement abnormality.

9. The intelligent motion control system for a cell injection device based on artificial intelligence according to claim 8, characterized in that: Once the abnormal state autonomous diagnosis and recovery module detects a confirmed abnormal event, it immediately sends an interrupt signal and an abnormality type code to the adaptive motion planning and decision-making center. The adaptive motion planning and decision-making center then calls the corresponding recovery protocol from the preset strategy library according to the anomaly type, or plans a new safe withdrawal trajectory and relocation trajectory in real time based on the current environmental state, and instructs the feedforward and feedback composite execution module to execute it.

10. The intelligent motion control system for a cell injection device based on artificial intelligence according to claim 1, characterized in that: The system operates within a hierarchical real-time control architecture, which includes a task management layer, an intelligent decision-making layer, and a real-time control layer. The task management layer operates on a second-level timescale and is responsible for task parsing, batch planning, and human-computer interaction. The intelligent decision-making layer operates on a millisecond-level timescale and encompasses the functions of adaptive motion planning, a decision-making center, and an abnormal state autonomous diagnosis and recovery module, performing online decision-making and planning. The real-time control layer operates on a microsecond-level timescale and includes the underlying control loop of the feedforward and feedback composite execution module, ensuring the execution of instructions. The three layers exchange data via a deterministic real-time communication bus.

Images