Microscopic imaging motion positioning intelligent control system based on artificial intelligence

Abstract

The present application relates to the technical field of micro-imaging motion positioning control, and particularly to a micro-imaging motion positioning intelligent control system based on artificial intelligence, which comprises the following technical solutions: a characteristic dynamic extraction unit is constructed, deep convolutional neural networks and graph neural networks are used to extract the appearance and geometric structure characteristic flow of a target in parallel, and robust representation of the multi-dimensional attributes of the target is formed; a multi-modal motion reasoning unit is designed, a gating cycle unit network is used to model the time sequence of the fused double-flow characteristics, and a Gaussian mixture model is used to express the displacement hypothesis distribution; an adaptive strategy generation unit is introduced, the control problem is converted into a reinforcement learning problem decided by a strategy network and a value network; a hierarchical real-time architecture is designed, the high-frequency low-delay deterministic control task and the low-frequency asynchronous learning optimization task are decoupled; the feature recognition and stability under the condition of dramatic changes in the morphology of the target are improved, and the defects of poor adaptability of the traditional fixed template matching method are overcome.

Description

Technical Field

[0001] This invention relates to the field of motion positioning control technology for microscopic imaging, and more particularly to an intelligent control system for motion positioning in microscopic imaging based on artificial intelligence. Background Technology

[0002] Artificial intelligence technology is playing an increasingly important role in image processing and automated control. Through deep learning models, it endows systems with powerful perception, analysis, and decision-making capabilities, and is widely used in various branches such as industrial inspection, autonomous driving, and biomedical imaging. Among them, microscopic imaging technology, as a key means of observing the microscopic world, relies heavily on the stability of the sample and the precision of the imaging process for its imaging quality and analytical accuracy.

[0003] Among them, the motion positioning and control system based on microscopic imaging aims to achieve automatic tracking of targets (such as cells and particles) and feedback control of platforms (such as stages) by analyzing image sequences in real time, so as to keep the target always in the center of the field of view or to follow a specific path. The basic principle of this technology involves extracting target features from consecutive image frames, calculating their positional offset, and generating corresponding control commands.

[0004] Existing technologies typically employ traditional image processing algorithms or basic machine learning models for target detection and localization. However, these methods have significant limitations when dealing with the complex dynamic scenes commonly encountered in microscopic imaging: First, traditional algorithms are poorly robust to image noise, illumination changes, and target morphological alterations (such as cell division and deformation), easily leading to tracking loss or localization drift; second, matching methods based on fixed feature templates struggle to adapt to non-rigid deformations or scale changes that occur during target motion, resulting in insufficient accuracy in the calculated motion vectors; third, existing control strategies are mostly simple proportional-integral-derivative (PCI) control with fixed parameters, unable to adaptively adjust to abrupt changes in target motion patterns (such as acceleration or change of direction), leading to system response lag or oscillations. These shortcomings are particularly prominent in life science research requiring long-term, high-precision dynamic observation, severely impacting the reliability of experimental data and the efficiency of automated processes.

[0005] Therefore, there is an urgent need for a system that can intelligently adapt to complex microscopic environments and achieve highly robust motion positioning and precise adaptive control. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent control system for motion positioning in microscopic imaging based on artificial intelligence, so as to solve the problems of poor positioning robustness, insufficient motion vector accuracy, and system response lag or oscillation caused by changes in target features, environmental interference and rigid control strategies when dealing with complex dynamic scenes in microscopic imaging.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] An AI-based intelligent control system for motion positioning in microscopic imaging includes an image sensing module, a collaborative perception and decision-making module, and a motion control execution module.

[0009] Among them, the image sensing module continuously acquires time-series image sequences under the microscopic field of view;

[0010] The collaborative perception and decision-making module is connected to the image sensing module to perform deep collaborative analysis on the input time-series image sequences, generating target motion state estimates and adaptive control decisions. The collaborative perception and decision-making module includes a feature dynamic extraction unit, a multimodal motion inference unit, and an adaptive policy generation unit. The feature dynamic extraction unit extracts the target's apparent feature stream and geometric structure feature stream in parallel from the current image frame and historical image frames. The multimodal motion inference unit is connected to the feature dynamic extraction unit, fusing the apparent feature stream and geometric structure feature stream and performing spatiotemporal context modeling to infer the target's true motion state. The adaptive policy generation unit is connected to the multimodal motion inference unit, generating the optimal control policy based on the motion inference results and the system's real-time performance indicators.

[0011] The motion control execution module is connected to the collaborative sensing and decision-making module. It receives control decisions and drives the actuators of the microscopic imaging platform to perform displacement, thereby achieving closed-loop tracking or path following of the target. The motion control execution module includes an instruction parsing unit, a drive unit, and a state feedback unit. The instruction parsing unit decodes the control decisions output by the adaptive strategy generation unit into a set of controller parameters and calculates the error between the target position setpoint and the current estimated position. The drive unit generates drive signals to drive the actuators based on the updated controller parameters and position error. The state feedback unit collects the actual displacement data of the actuators in real time, and after filtering, feeds back the actual pose information of the platform and the tracking error statistics to the adaptive strategy generation unit.

[0012] Furthermore, in the dynamic feature extraction unit, the apparent feature flow is extracted through a deep convolutional neural network. The input of this network is an image region centered on the historical predicted location, and the output is a 512-dimensional feature vector. The geometric structure feature flow is extracted through a graph neural network. This network constructs a dynamic graph structure of key points within the target region. The node features are the coordinates and local descriptors of the key points, and the edge features are the spatial relationships between the key points. The features of the nodes and edges are iteratively updated through three layers of graph convolution, and finally aggregated to generate a 256-dimensional geometric feature vector.

[0013] Furthermore, the deep convolutional neural network adopts a residual network structure and connects a spatial transformation layer at the end of the network; the spatial transformation layer performs affine transformation alignment on the input image region based on the displacement estimate output by the multimodal motion inference unit in the previous frame.

[0014] Furthermore, the multimodal motion inference unit performs the following process: concatenating the appearance feature vector and geometric feature vector extracted from the current frame with the corresponding feature vector from the previous frame to form a fused feature; inputting the fused feature into a gated recurrent unit network with 128 hidden units; based on the current fused feature and historical state, the gated recurrent unit network outputs a hypothetical distribution of the target's displacement in the current frame relative to the previous frame, which is represented by a two-dimensional Gaussian mixture model containing three Gaussian components, each of which contains a mean vector, a covariance matrix, and mixture weights.

[0015] Furthermore, in the gated recurrent unit network, the activation functions for the update gate and reset gate are Sigmoid functions, and the activation function for the candidate hidden state is hyperbolic tangent function. At each time step, the network calculates the values ​​of the update gate and reset gate based on the current fused features and the previous hidden state. The reset gate is used to control the degree of forgetting of the previous hidden state, and the candidate hidden state is calculated in combination with the current input. The update gate is used to perform weighted fusion of the previous hidden state and the candidate hidden state to obtain the current hidden state and the output.

[0016] Furthermore, the adaptive policy generation unit comprises a policy network and a value network. The inputs to the policy network include the hidden states of the gated recurrent unit network, the statistical characteristics of the current displacement hypothesis distribution, and the sliding window mean of the actual platform displacement and the system tracking error fed back by the motion control execution module. The policy network is a fully connected neural network with two hidden layers, each with 64 neurons. The output layer outputs a probability distribution in the discrete action space through the Softmax function. The action space is defined as an incremental adjustment combination of proportional gain, integral gain, and differential gain, with a total of 27 discrete actions. The value network has the same input and hidden layer structure as the policy network, and the output layer is a single scalar.

[0017] Furthermore, the adaptive policy generation unit updates the parameters of the policy network and the value network online through the proximal policy optimization algorithm. The reward function of the proximal policy optimization algorithm is designed as a weighted sum of four components: the first component is the negative normalized tracking error, the second component is the control action smoothness reward, the third component is the system stability reward, and the fourth component is the exploration reward. The weights of the four components are 0.7, 0.15, 0.1, and 0.05, respectively.

[0018] Furthermore, the filter in the state feedback unit adopts an extended Kalman filter, and its state vector includes the position, velocity and acceleration of the platform; the process model is based on the assumption of uniform acceleration motion, and the observation model is based on the readings of the grating ruler or laser interferometer; the extended Kalman filter performs a prediction and update cycle every 1 millisecond.

[0019] Furthermore, the drive unit generates a pulse width modulation signal or an analog voltage signal based on the updated proportional-integral-derivative parameters and position error, which drives the piezoelectric ceramic actuator or stepper motor actuator.

[0020] Furthermore, the system is deployed in a hierarchical real-time architecture, which includes a high-frequency control loop and a low-frequency learning loop. The high-frequency control loop operates on a period of at least 100 Hz, based on the frame rate of the image sensing module, and executes the entire process from image acquisition, dynamic feature extraction, multimodal motion inference to instruction parsing and drive output. The low-frequency learning loop is triggered at fixed time intervals or when the tracking performance index is below a threshold, with a period of at least 1 second, and executes parameter updates for the policy network and value network in the adaptive policy generation unit, as well as adjustments to the neural network in the dynamic feature extraction unit.

[0021] Compared with the prior art, the beneficial effects of the present invention are:

[0022] 1. This invention constructs a dynamic feature extraction unit and uses deep convolutional neural networks and graph neural networks in parallel to extract the apparent and geometric structural feature flow of the target, forming a robust representation of the target's multi-dimensional attributes. In particular, the graph neural network's ability to process dynamic graph structures enables it to effectively capture the inherent geometric constraints of non-rigid changes such as cell division and deformation, fundamentally improving the feature recognition and stability under conditions of drastic changes in target morphology, and overcoming the poor adaptability of traditional fixed template matching methods.

[0023] 2. This invention designs a multimodal motion inference unit and uses a gated recurrent unit network to perform temporal modeling of the fused dual-stream features, and expresses the displacement hypothesis distribution with a Gaussian mixture model. It not only integrates the multi-source information of the current frame, but also memorizes and utilizes historical motion patterns, which can effectively distinguish between real motion and apparent changes caused by noise and illumination fluctuations, thereby outputting a high-precision motion vector estimate with probability confidence, which significantly reduces the risk of positioning drift.

[0024] 3. This invention transforms the control problem into a reinforcement learning problem involving collaborative decision-making by a policy network and a value network by introducing an adaptive policy generation unit. The system can dynamically optimize the parameters of the proportional-integral-derivative controller online based on performance indicators such as real-time tracking error and platform status. This learning-based adaptive mechanism enables the control system to proactively adapt to sudden changes in the target motion mode, such as acceleration and sharp turns, achieving forward-looking adjustment. This effectively eliminates the response lag and oscillation problems caused by fixed-parameter proportional-integral-derivative control, improving the dynamic tracking performance and overall stability of the system.

[0025] 4. This invention decouples the high-frequency, low-latency deterministic control task from the low-frequency, asynchronous learning and optimization task by designing a layered real-time architecture. This ensures that the system meets the hard requirements of real-time control for microscopic imaging while also utilizing computational gaps for continuous model optimization and self-evolution. This enables the system to maintain and improve its performance during long-term deployment, achieving a balance between real-time performance and intelligence. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall technical solution architecture of the intelligent control system for motion positioning in microscopic imaging based on artificial intelligence proposed in this invention.

[0027] Figure 2 This is a schematic diagram of the core principle framework of the collaborative perception and decision-making module in this invention;

[0028] Figure 3 This is a flowchart illustrating the logical flow of the feature dynamic extraction and multimodal motion reasoning stages in this invention.

[0029] Figure 4 This is a schematic diagram of the multi-level interaction relationship and data flow between adaptive strategy generation and motion control execution in this invention;

[0030] Figure 5 This is a schematic diagram illustrating the interaction between the high-frequency control loop and the low-frequency learning loop in the hierarchical real-time architecture of this invention. Detailed Implementation

[0031] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0032] Example 1:

[0033] The intelligent control system for motion positioning in microscopic imaging based on artificial intelligence provided by this invention has the following overall technical architecture: Figure 1As shown, this system aims to achieve robust and precise real-time tracking and positioning of dynamic targets under microscopic vision, such as living cells, microorganisms, or micro / nano manipulator probes. The core of the system consists of an image sensing module, a collaborative perception and decision-making module, and a motion control execution module. These three modules are connected in series via a high-speed data bus and a real-time communication protocol, forming a complete closed loop from perception to decision-making to execution.

[0034] The image sensing module is the system's perception front end, with its core components being a scientific-grade complementary metal-oxide-semiconductor (CMOS) camera or an electron-multiplying charge-coupled device (CCD) camera. This camera couples to the microscope's imaging optical path via an optical interface, continuously acquiring temporal image sequences under microscopic conditions at a rate of at least 100 frames per second. Each frame is a grayscale or color digital matrix with a specific depth. The image sensing module integrates an image preprocessing unit, which performs a series of standardized operations on the raw image sequence to improve the stability of subsequent processing. These operations include fixed-pattern noise correction, i.e., pixel-level gain and offset compensation of the raw image using pre-stored dark and flat field images; non-uniform illumination correction, i.e., applying Gaussian filtering-based background estimation and division operations to eliminate illumination gradients; and dynamic range stretching, i.e., linearly mapping image pixel values ​​to a preset full-scale range, for example, mapping from 12-bit raw data to an 8-bit representation range of 0 to 255. The preprocessed image sequence is streamed in real-time to the collaborative sensing and decision-making module in the form of data packets via Gigabit Ethernet or the camera link interface. Each data packet contains a frame sequence number, timestamp, image data matrix, and checksum to ensure the integrity and timing accuracy of data transmission.

[0035] The collaborative perception and decision-making module is the intelligent core of the system, responsible for deep analysis of the input time-series image sequences and generating control decisions. Its core principle framework is as follows: Figure 2 As shown. This module is deployed on a high-performance embedded computing platform equipped with a graphics processing unit, running a real-time operating system to ensure deterministic task scheduling. The module is further subdivided into a feature dynamic extraction unit, a multimodal motion inference unit, and an adaptive policy generation unit. These three units are logically connected sequentially, and the data flow strictly follows the attached... Figure 2 The path shown.

[0036] The feature dynamic extraction unit receives a preprocessed image stream from the image sensing module. Its primary task is to extract the appearance feature stream and geometric structure feature stream of the target in parallel from the current image frame and associated historical image frames. The logical flow framework of this stage is as follows: Figure 3As shown. For the extraction of appearance feature streams, a deep convolutional neural network is deployed inside the unit. This network adopts a residual network structure, specifically a variant of the residual network with 18 weight layers. The input to the network is not the entire frame image, but an image region dynamically cropped centered on the historical prediction position. The size of this region is adaptively determined based on the estimated size of the target in the previous frame, typically a safe region extending 20% ​​to 50% beyond the target bounding box. Before being input into the network, this region image passes through a spatial transformation layer. The spatial transformation layer performs an affine transformation on the input region based on the displacement estimate output by the multimodal motion inference unit in the previous frame to achieve a coarse alignment with the target pose in the previous frame. This aims to mitigate the interference of motion blur caused by the rapid movement of the target on feature extraction. The spatially transformed and aligned image patch is scaled to a fixed size of 224 pixels by 224 pixels and then fed into the deep convolutional neural network. The front end of the network consists of stacked convolutional layers, batch normalization layers, and modified linear unit activation functions, used to extract hierarchical visual features from low to high levels. The network terminates at a global average pooling layer and a fully connected layer, ultimately outputting a 512-dimensional real-valued feature vector. This 512-dimensional vector highly encodes the apparent attributes of the target from the current viewpoint, such as texture details, color distribution, and local contrast patterns, and exhibits a certain degree of robustness to changes in illumination and partial occlusion.

[0037] For the extraction of geometric structural feature flow, the dynamic feature extraction unit simultaneously runs a graph neural network processing flow. This flow begins with keypoint detection and description of the same input image region, i.e., the region after spatial transformation and alignment. A scale-invariant feature transformation algorithm or an accelerated segmented test feature detection algorithm is used to extract no fewer than 50 highly repetitive keypoints from the image region. Each keypoint is described by its pixel coordinates, scale, and orientation angle parameters. Subsequently, a 128-dimensional local binary feature descriptor, such as an oriented gradient histogram descriptor, is calculated for each keypoint. At this point, each keypoint is represented as a node containing spatial coordinates and a local descriptor. Next, the unit constructs a dynamic graph structure among these keypoint nodes. The set of nodes in the graph represents all detected keypoints, and the initial feature vector of each node is the concatenation of its coordinates and descriptor. The set of edges in the graph is constructed using the K-nearest neighbor algorithm, i.e., for each node, its five nearest neighbor nodes are found in Euclidean space, and undirected edges are established to connect them. The initial features of each edge are defined as the normalized direction vector and distance scalar connecting the two nodes. This dynamic graph structure is fed into a three-layer graph convolutional neural network. Each graph convolutional operation performs message passing, aggregation, and updating. Specifically, for each node, the layer first aggregates the transformed feature information from all its neighboring nodes. The aggregation function is a weighted sum, with weights generated by edge features through a learnable linear layer. The aggregated neighbor information is concatenated with the node's own features, and then the node's feature representation is updated through a fully connected layer and a non-linear activation function. Simultaneously, the edge features are also updated based on the new features of the nodes connecting the two ends. After three such iterations, the features of both nodes and edges are fully refined. Finally, the unit aggregates the updated features of all nodes into a fixed-length 256-dimensional geometric feature vector through a global graph pooling operation, such as element-wise max pooling of all node features. This 256-dimensional vector deeply encodes the target's contour shape, skeletal structure, keypoint topological relationships, and potential deformation patterns, which is crucial for maintaining feature consistency when the target undergoes non-rigid changes such as stretching, bending, or splitting.

[0038] The multimodal motion inference unit is closely connected to the feature dynamic extraction unit. Its core responsibility is to fuse dual-stream features and perform spatiotemporal context modeling to infer the true motion state of the target. The unit receives the current frame's apparent feature vector and geometric feature vector from the feature dynamic extraction unit. Before temporal fusion, the unit maintains a 10-fold first-in-first-out buffer to store the corresponding feature vector pairs from the last 10 frames. For the current frame, the unit performs the following operations: First, it concatenates the 512-dimensional apparent feature vector and the 256-dimensional geometric feature vector extracted from the current frame along the feature dimensions to obtain a 768-dimensional fused feature vector. Next, it retrieves the corresponding 768-dimensional fused feature vector from the previous frame from the buffer. It then concatenates the current frame's fused feature vector and the previous frame's fused feature vector again along the feature dimensions to form a 1536-dimensional extended fused feature vector. This 1536-dimensional vector simultaneously contains the multimodal information of the current frame and the corresponding information of the previous frame, providing a direct inter-frame comparison basis for motion inference.

[0039] Subsequently, this 1536-dimensional extended fusion feature vector is input into a gated recurrent unit (GRU) network. This GRU network has 128 hidden units, whose internal hidden states serve as the system's short-term memory, encoding the historical patterns and trends of the target's motion. At each time step, the network calculates update and reset gates based on the current input and the previous hidden state. The activation functions for the update and reset gates are sigmoid functions, with output values ​​between 0 and 1, controlling the degree to which the current hidden state retains information from the previous hidden state and the degree to which information from the previous hidden state is forgotten when calculating candidate states, respectively. The activation function for the candidate hidden states is the hyperbolic tangent function. The specific calculation process is as follows: First, the network calculates the update and reset gate vectors based on the current input and the previous hidden state, using a learnable weight matrix and bias. Then, the reset gate vector is used to perform element-wise multiplication on the previous hidden state to control the inflow of historical information, and then combined with the current input to calculate the candidate hidden state. Finally, using the updated gate vector as weights, a weighted sum is applied to the previous hidden state and the candidate hidden states to obtain the hidden state and network output at the current time step. The output of the gated recurrent unit network and the updated hidden state are used together for subsequent inference.

[0040] The core output of the multimodal motion inference unit is a probability distribution of the target's displacement hypothesis in the current frame relative to the previous frame. This distribution is represented by a two-dimensional Gaussian mixture model, where the two dimensions correspond to the X-axis and Y-axis displacements on the image plane. This Gaussian mixture model contains three Gaussian components. Each Gaussian component is defined by three parameters: a two-dimensional mean vector representing the center of the displacement hypothesis; a 2×2 covariance matrix representing the uncertainty or spread of the displacement hypothesis; and a scalar mixture weight representing the importance of the component in the overall mixture model. The output of the gated recurrent unit network passes through a specific projection layer, which maps the 128-dimensional hidden state to all the parameters required for these three Gaussian components. Specifically, the projection layer outputs a 27-dimensional vector, where the first six dimensions are transformed into the three mean vectors, the middle 12 dimensions are transformed positive definitely into the upper triangular elements of the three covariance matrices, and the last three dimensions are normalized using the Softmax function and used as the mixture weights. Thus, the system obtains a complete displacement probability distribution, rather than just a single-point estimate. This Gaussian mixture model can effectively distinguish between displacement patterns generated by real motion and spurious apparent changes caused by image noise, local occlusion, or lighting flicker, because the latter usually exhibit high uncertainty or multimodal conflict in probability distribution.

[0041] The adaptive policy generation unit is a direct downstream unit of the multimodal motion inference unit. Its function is to generate the optimal control policy for adjusting the motion control execution module based on the motion inference results and the system's real-time performance indicators. The interaction between this unit and the motion control execution module is as follows: Figure 4 As shown, the adaptive policy generation unit comprises two core neural networks: a policy network and a value network. The input to the policy network is a highly integrated state vector. This state vector is composed of five parts: the first part is the current 128-dimensional hidden state from the gated recurrent unit network, encoding the motion history context. The second part is the statistical characteristics of the current displacement hypothesis Gaussian mixture model, including the weighted sum of three mean vectors, the weighted average of the traces of the covariance matrices of each component, and the entropy of the mixture weights, totaling seven scalars. The third part is the sliding window mean of the actual platform displacement over the last 20 control cycles, fed back from the motion control execution module, which is a two-dimensional vector. The fourth part is the sliding window mean of the system tracking error over the last 20 control cycles, defined as the Euclidean distance between the estimated target position and the actual platform position, which is a scalar. The fifth part is the parameter set of the proportional-integral-derivative controller in the current motion control execution module, including proportional gain, integral gain, and derivative gain, totaling three scalars. All of these parts together constitute a 141-dimensional state vector.

[0042] The policy network is a fully connected neural network with two hidden layers. The input layer receives a 141-dimensional state vector. The first hidden layer contains 64 neurons using a modified linear unit activation function (MRU). The second hidden layer also contains 64 neurons, also using the MRU. The output layer is connected to a softmax function, outputting a 27-dimensional probability distribution vector. These 27 dimensions correspond to a discrete action space. The action space is defined as all possible combinations of incremental adjustments to the proportional gain, integral gain, and derivative gain. Each gain adjustment has three possibilities: increase by a fixed step, decrease by a fixed step, or remain unchanged. Therefore, there are a total of 3 × 3 × 3 = 27 discrete actions. The probability distribution output by the policy network represents the recommended probability of taking each parameter adjustment action in the current system state.

[0043] The value network has the exact same input and hidden layer structure as the policy network, i.e., it takes a 141-dimensional state as input and passes through two hidden layers of 64 neurons each. However, its output layer is different. The value network's output layer is a single neuron, and it does not use Softmax, directly outputting a scalar value. This scalar value represents the value network's evaluation of the current system state, that is, the estimated value of the expected cumulative reward starting from the current state.

[0044] The adaptive policy generation unit employs a proximal policy optimization algorithm as its core learning mechanism to update the parameters of the policy network and value network online. This algorithm runs in a separate, low-frequency learning thread. Its goal is to maximize the cumulative reward gained by the system over long-term operation while ensuring that the magnitude of each policy update is not too large to maintain learning stability. The design of the reward function is crucial, as it guides the direction of policy optimization. The reward function of this system consists of four linearly weighted components. The first component is the negative normalized tracking error, calculated by dividing the current tracking error by a preset maximum allowable error; this component has a weight of 0.7, directly encouraging the reduction of tracking deviation. The second component is the control action smoothness reward, calculated by the negative norm of the parameter change corresponding to the current action and the previous action; this component has a weight of 0.15, aiming to suppress drastic fluctuations in control parameters. The third component is the system stability reward; when the combined output of the proportional-integral-derivative controller does not reach the saturation limit of the actuator, a fixed positive reward is given; this component has a weight of 0.1, encouraging the system to operate in the linear stable region. The fourth component is the exploration reward, which is the entropy of the probability distribution output by the policy network multiplied by a small coefficient with a weight of 0.05. It aims to encourage the policy to maintain a certain degree of exploration and avoid premature convergence to a local optimum.

[0045] The objective function of the proximal policy optimization algorithm can be expressed as:

[0046]

[0047] in, Let θ be the cutoff objective function of the policy network; θ represents the parameters of the policy network. Is it a matter of new and old strategies in a state? Select action The probability ratio, The new strategy selects an action with parameter θ. The probability, It represents the probability of the old strategy selecting the same action before the update; It is the advantage function estimated by the value network, used to evaluate the performance of the current action relative to the average performance; ε is the clipping range hyperparameter, which is set to 0.2 in this system; clip(x,a,b) is the clipping function, which restricts x to the interval [a,b]; min(·) is the minimum value function; [·] represents the expected value estimate for time step t. The core of this formula lies in its pruning mechanism, which restricts the probability ratio. Within the interval [1-ε, 1+ε], the magnitude of each policy update is constrained, ensuring the stability of the training process. The value network updates by minimizing the mean squared error between its output and the actual reward. The parameter updates of the policy network and the value network are triggered by a low-frequency learning loop, without interfering with the real-time operation of the high-frequency control loop.

[0048] The motion control execution module is the physical execution end of the system, responsible for translating intelligent decisions into precise mechanical displacements. This module includes an instruction parsing unit, a high-precision drive unit, and a state feedback unit. The instruction parsing unit receives action numbers output by the adaptive strategy generation unit. Internally, the unit maintains an action encoding table, mapping 27 action numbers to specific proportional gain increments, integral gain increments, and derivative gain increments in triplets. For example, action number 5 might correspond to a proportional gain increase of 0.1, an integral gain decrease of 0.05, and a unchanged derivative gain. Based on the current proportional-integral-derivative parameters and the action mapping result, the instruction parsing unit calculates a new parameter set. Simultaneously, this unit obtains the optimal value of the current target displacement estimate from the multimodal motion inference unit, typically taking the displacement vector after weighted averaging in a Gaussian mixture model, and adds it to the actual platform position provided by the state feedback unit to obtain the target position setpoint for the new control cycle. Subsequently, the position error between the target setpoint and the actual platform position is calculated.

[0049] The drive unit performs closed-loop control calculations based on the updated proportional-integral-derivative (PI-DE) parameters and the calculated position error. The PI-DE controller operates at a fixed frequency of at least 1 kHz. The controller calculates the proportional, integral, and derivative terms separately based on the position error. The integral term is obtained by accumulating the product of the historical error and the integral gain, and includes an anti-saturation limiting mechanism. The derivative term is calculated using a differential approximation of the error and then low-pass filtered to suppress high-frequency noise. The sum of these three terms yields the final control output. This output is converted into a corresponding drive signal depending on the actuator type. For nanometer-level positioning, piezoelectric actuators are typically driven, and the control output is converted into a high-resolution analog voltage signal ranging from 0 to 10 volts, output through a high-stability digital-to-analog converter. For micrometer-level positioning, stepper motors or voice coil motors are typically driven, and the control output is converted into pulse-width modulation (PWM) signals or current signals of the corresponding frequency and number. The drive signal is amplified by a power amplifier and applied to the actuator, thereby driving the microscope stage or objective lens to perform precise displacement.

[0050] The state feedback unit is a key sensor component for closed-loop control and closed-loop learning. This unit includes high-precision position sensors, such as grating rulers or laser interferometers, with resolution down to the nanometer level; and speed and current sensors integrated into the actuator. The unit acquires raw data such as the actual displacement, instantaneous velocity, and drive current of the actuator in real time at a sampling rate of at least 10 kHz. This raw data contains noise. To obtain the optimal estimate, the state feedback unit runs an extended Kalman filter. The filter's state vector is defined as 6-dimensional, including the platform's position, velocity, and acceleration in the X and Y axes. The process model is based on the assumption of discrete-time uniformly accelerated motion. The observation model is based on the readings of the position sensors; since the observation equations may be nonlinear with respect to the state, an extended Kalman filter is used for linearization. The filter executes a prediction and update cycle with a strict 1-millisecond period: in the prediction step, the current state and covariance are predicted based on the process model and the previous state estimate; in the update step, the predicted value is corrected using the latest sensor observations to obtain the optimal state estimate. The filtered actual pose information of the platform, including position and velocity, is provided in real time to the command parsing unit for error calculation. Meanwhile, the state feedback unit also calculates the statistics of recent tracking errors, such as the mean and variance of the sliding window, and packages these performance indicators together with the actual displacement of the platform and feeds them back to the adaptive strategy generation unit to form a complete closed-loop learning circuit.

[0051] To ensure that the system meets hard real-time control requirements while also enabling intelligent background optimization, this invention deploys the system in a layered real-time architecture, with the following interaction relationships: Figure 5As shown, the architecture is clearly divided into two levels: a high-frequency control loop and a low-frequency learning loop. The high-frequency control loop is the backbone of the system, and its cycle is strictly synchronized with the frame rate of the image sensing module, not lower than 100 Hz. Within each control cycle, the system must strictly complete the entire chain of tasks from image acquisition, preprocessing, dynamic feature extraction, multimodal motion inference, instruction parsing to drive signal output in strict sequence. All tasks are assigned the highest real-time priority, and their deadlines are guaranteed by the scheduler of the real-time operating system. Any timeout of any task will trigger a system alarm and enter a safe fault-tolerant mode. The high-frequency control loop ensures the ability to respond instantly to target motion.

[0052] The low-frequency learning loop runs in a separate, low-priority computation thread. This learning loop is triggered in two modes: fixed-time interval triggering, with a period of no less than 1 second; or performance triggering, i.e., triggering immediately when the average tracking error sliding window value of multiple consecutive control cycles exceeds a preset threshold. After triggering, the learning loop performs the following asynchronous tasks: First, it samples a batch of historical state-action-reward data from the experience replay buffer. Then, using this data, it performs one or more iterations of the proximal policy optimization algorithm, updating the parameters of the policy network and value network in the adaptive policy generation unit. Simultaneously, the learning loop may also use newly acquired image data and corresponding annotation information to fine-tune the deep convolutional neural network and graph neural network in the feature dynamic extraction unit to adapt to the specificity of the current observation target. All learning and optimization tasks must be completed before the next trigger cycle arrives, and their execution never preempts the computational resources of the high-frequency control loop. This decoupled design enables the system to possess both deterministic real-time control performance and continuous autonomous evolution capabilities.

[0053] In summary, this embodiment details the complete implementation plan of an artificial intelligence-based intelligent control system for motion positioning in microscopic imaging. The system acquires raw visual information through an image sensing module; it achieves an intelligent processing chain from robust feature extraction and probabilistic motion inference to adaptive decision-making through the feature dynamic extraction unit, multimodal motion inference unit, and adaptive strategy generation unit in the collaborative perception and decision-making module; and finally, it completes closed-loop control through precise driving and state feedback from the motion control execution module. Under the coordination of a hierarchical real-time architecture, the entire system achieves a unified system of high-precision positioning, robust tracking, and continuous self-optimization, providing a reliable technical foundation for automatic observation and operation in complex dynamic microscopic scenes.

[0054] Example 2:

[0055] For hardware system selection and integration: The image sensing module uses a scientific-grade CMOS or EMCCD camera with a frame rate ≥100Hz, supports external triggering, and features high signal-to-noise ratio and low noise characteristics. The computing platform uses a GPU-embedded industrial computer (such as NVIDIA Jetson AGX Orin) or an industrial computer with a dedicated GPU to run deep neural network models, requiring support for CUDA and a real-time operating system (such as Linux with PREEMPT_RT). The motion control execution module uses a piezoelectric ceramic stage, stepper motor, or voice coil motor driver with a resolution ≤1nm (piezoelectric) or ≤1µm (stepper), supporting analog voltage or PWM control. The position feedback sensor uses a grating ruler, laser interferometer, or encoder, with a resolution matching the actuator and a data update rate ≥10kHz. The communication interface uses Gigabit Ethernet, Camera Link, PCIe, EtherCAT, etc., ensuring low-latency data transmission and supporting real-time control protocols. The microscopic imaging platform uses an inverted or upright microscope, equipped with a motorized stage and objective lens, compatible with the selected camera and actuator, and with a stable optical system.

[0056] For software system construction and training: In the system software architecture, the operating system adopts Linux + PREEMPT_RT real-time kernel, and the development environment adopts Python + PyTorch / TensorFlow + ROS2 for inter-module communication. The high-frequency control loop is ≥100 Hz, synchronized with the image frame rate, and the low-frequency learning loop is ≥1Hz, asynchronously updating the policy network. In neural network model training and deployment, the input of the deep convolutional neural network (ResNet) is a 224×224 image patch, and the output is a 512-dimensional feature vector. The training data is a microscopic image dataset containing different target shapes and lighting conditions. The input of the graph neural network (GCN) is a keypoint graph structure, and the output is a 256-dimensional geometric feature vector. The training data is a sequence of microscopic images with labeled keypoints. The input of the GRU network is a fused feature vector, and the output is a displacement distribution (Gaussian mixture model). The training data is time-series image pairs + real displacement annotations. The input of the policy network and value network (PPO) is the system state vector, and the output is action probability and state value. The training data is simulation environment or real system interaction data. In the training process: pre-training trains the feature extraction network on publicly available microscopic datasets or simulation data; transfer learning fine-tunes the model using images actually acquired in experiments; online reinforcement learning runs the PPO algorithm on a real system to progressively optimize the control strategy.

[0057] The experimental procedure includes system initialization and calibration, as well as target tracking. System initialization and calibration include camera calibration (intrinsic parameters, distortion correction), stage motion calibration (stepping / piezoelectric response curves), grating ruler / laser interferometer zero-point calibration, and image preprocessing parameter settings (such as illumination correction and noise filtering). Target tracking includes: placing the sample, adjusting the microscope focal length and illumination; starting the image acquisition system, setting the acquisition area and frame rate; selecting or clicking the target in the software interface to start tracking; the system automatically runs the high-frequency control loop to adjust the stage in real time; the low-frequency learning loop asynchronously updates network parameters in the background; recording the tracking trajectory, error curve, and changes in control parameters; and exporting the data for subsequent analysis after the experiment.

[0058] The core feature extraction employs a dual-stream parallel approach to handle deformation. The superficial feature stream extraction network is a deep convolutional neural network (such as ResNet), taking an image region centered on a historical location (aligned via spatial transformation) as input and outputting a 512-dimensional feature vector. To handle irregular motion, it captures subtle changes in cell surface texture, edges, and local contrast; for example, the texture changes of local protrusions and pseudopodia formed during phagocytosis. The alignment mechanism (spatial transformation layer) reduces motion blur caused by rapid deformation, ensuring feature stability. The geometric structure feature stream extraction network is a graph neural network, taking a dynamic graph of keypoints as input (nodes = keypoint coordinates + descriptors, edges = spatial relationships) and outputting a 256-dimensional geometric feature vector. To handle irregular motion, it captures topological changes, dynamically altering the relative positions and connections between keypoints during phagocytosis and division; it models non-rigid deformation, passing spatial constraints between keypoints through graph convolution to learn deformation patterns; and it maintains feature consistency, ensuring continuity of geometric structure features even with drastic changes in cell shape.

[0059] The motion inference mechanism performs temporal fusion and probabilistic modeling. Multimodal feature fusion concatenates the appearance features (512-dimensional) and geometric features (256-dimensional) of the current frame into a 768-dimensional fused feature, which is then concatenated with the fused features of the previous frame into a 1536-dimensional extended feature, providing direct inter-frame comparison. Temporal context modeling uses a gated recurrent unit (GRU) network to process the extended feature sequence. The hidden states of the GRU act as "short-term memory," encoding historical patterns and trends of cell motion, such as temporal dependencies like acceleration patterns and directional mutations during phagocytosis. The probability distribution output of the displacement hypothesis is a two-dimensional Gaussian mixture model, containing three Gaussian components. Each component includes the mean (displacement direction and magnitude), covariance (uncertainty), and mixture weights (confidence). When dealing with irregular motion, it expresses a multi-modal attitude quantity, absorbing multiple possible motion hypotheses (such as extending in different directions simultaneously). The GMM can represent multiple possibilities. It distinguishes between noise and real motion. Spurious changes caused by image noise or local occlusion will show high uncertainty or low weight in the probability distribution. It provides confidence estimation. The system not only outputs "where it is going" but also "how confident it is", which facilitates subsequent control decisions.

[0060] In adaptive control, the policy is adjusted based on motion uncertainty. The policy network input includes motion uncertainty, and the input state vector includes statistical characteristics of displacement distribution (such as weighted average displacement, covariance trace, and entropy). High entropy indicates high motion uncertainty (such as unclear direction during swallowing). The policy network can adjust the balance between exploration and utilization based on uncertainty. Reinforcement learning reward functions guide adaptation. The reward function includes tracking error, motion smoothness, stability, and exploration rewards. Faced with irregular motion, the system encourages trying new control strategies through exploration rewards to avoid getting trapped in local optima. A hierarchical real-time architecture ensures stable response. The high-frequency control loop performs fast responses based on the current best estimate (such as the weighted mean of GMM). In the low-frequency learning loop, when tracking performance deteriorates (such as swallowing leading to increased error), it is triggered to update network parameters and adaptively adjust feature extraction and motion inference models.

[0061] The above specific embodiments are merely several preferred embodiments of the present invention. Based on the technical solutions of the present invention and the relevant teachings of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.

Claims

1. An artificial intelligence-based intelligent control system for microscopic imaging motion positioning, characterized in that: The system comprises an image sensing module, a cooperative perception and decision module, and a motion control execution module. The image sensing module continuously collects a time sequence of images under a microscopic field of view. The cooperative perception and decision module is connected to the image sensing module, performs deep cooperative analysis on the input time sequence of images, and generates target motion state estimation and adaptive control decisions. The cooperative perception and decision module comprises a feature dynamic extraction unit, a multi-modal motion inference unit, and an adaptive strategy generation unit. The feature dynamic extraction unit extracts a target appearance feature stream and a geometric structure feature stream from a current image frame and a historical image frame in parallel. The multi-modal motion inference unit is connected to the feature dynamic extraction unit, fuses the appearance feature stream and the geometric structure feature stream, and performs spatio-temporal context modeling to infer the real motion state of the target. The adaptive strategy generation unit is connected to the multi-modal motion inference unit, generates an optimal control strategy based on the motion inference result and real-time system performance indicators. The motion control execution module is connected to the cooperative perception and decision module, receives control decisions, and drives the actuators of the microscopic imaging platform to displace, thereby realizing closed-loop tracking or path following of the target. The motion control execution module comprises an instruction analysis unit, a driving unit, and a state feedback unit. The instruction analysis unit decodes the control decisions output by the adaptive strategy generation unit into controller parameter sets, and calculates the error between the target position set point and the current estimated position. The driving unit generates driving signals to drive the actuators based on the updated controller parameters and the position error. The state feedback unit collects actual displacement data of the actuators in real time, filters the data, and feeds back the actual pose information of the platform and tracking error statistics to the adaptive strategy generation unit. In the feature dynamic extraction unit, the appearance feature stream is extracted by a deep convolutional neural network, the input of which is an image region centered on the historical predicted position, and the output is a 512-dimensional feature vector. The geometric structure feature stream is extracted by a graph neural network, which constructs the key points in the target region into a dynamic graph structure, with node features being the coordinates and local descriptors of the key points, and edge features being the spatial relationships between the key points. The deep convolutional neural network adopts a residual network structure and is connected with a spatial transformation layer at the end of the network. The spatial transformation layer performs affine transformation alignment on the input image region based on the displacement estimation output by the multi-modal motion inference unit in the previous frame. The multi-modal motion inference unit performs the following process: splicing the appearance feature vector and the geometric feature vector extracted from the current frame with the corresponding feature vectors of the previous frame to form a fusion feature; inputting the fusion feature into a gated recurrent unit network with 128 hidden units; and based on the current fusion feature and historical state, outputting a displacement hypothesis distribution of the target relative to the previous frame in the current frame, which is represented by a two-dimensional Gaussian mixture model and includes three Gaussian components, each component including a mean vector, a covariance matrix, and a mixing weight. In the gated recurrent unit network, the activation function of the update gate and the reset gate adopts the Sigmoid function, and the activation function of the candidate hidden state adopts the hyperbolic tangent function; at each time step, the network calculates the values of the update gate and the reset gate according to the current fusion feature and the previous hidden state; the reset gate is used to control the forgetting degree of the previous hidden state, and the candidate hidden state is calculated in combination with the current input; the update gate is used to weight and fuse the previous hidden state and the candidate hidden state to obtain the current hidden state and the output. 2.The artificial intelligence-based intelligent control system for motion positioning in microscopic imaging according to claim 1, wherein: The adaptive policy generation unit includes a policy network and a value network; the input of the policy network includes the hidden state of the gated recurrent unit network, the statistical features of the current displacement hypothesis distribution, and the sliding window mean values of the actual displacement of the platform and the system tracking error fed back by the motion control execution module; the policy network is a fully connected neural network with two hidden layers, each layer having 64 neurons, and the output layer outputs a probability distribution on a discrete action space through a Softmax function, the action space being defined as a combination of incremental adjustments of proportional gain, integral gain and derivative gain, and there are 27 discrete actions; the value network has the same input layer and hidden layer structure as the policy network, and the output layer is a single scalar. 3.The artificial intelligence-based intelligent control system for motion positioning in microscopic imaging according to claim 2, wherein: The adaptive policy generation unit updates the parameters of the policy network and the value network online through the proximal policy optimization algorithm; the reward function of the proximal policy optimization algorithm is designed as the weighted sum of four components, the first component being the negative normalized tracking error, the second component being the control action smoothness reward, the third component being the system stability reward, and the fourth component being the exploration reward; the weights of the four components are 0.7, 0.15, 0.1 and 0.05 respectively. 4.The artificial intelligence-based intelligent control system for motion positioning in microscopic imaging according to claim 1, wherein: The filter in the state feedback unit adopts an extended Kalman filter, and its state vector includes the position, velocity and acceleration of the platform; the process model is based on the assumption of uniform acceleration, and the observation model is based on the readings of the grating ruler or the laser interferometer; the extended Kalman filter performs a prediction and update cycle every 1 millisecond. 5.The artificial intelligence-based intelligent control system for motion positioning in microscopic imaging according to claim 1, wherein: The drive unit generates a pulse width modulation signal or an analog voltage signal based on the updated proportional-integral-derivative parameters and the position error, and drives the piezoelectric ceramic actuator or the stepper motor actuator.

6. The artificial intelligence-based intelligent control system for microscopic imaging motion positioning according to claim 1, wherein: The system is deployed in a hierarchical real-time architecture, which includes a high-frequency control loop and a low-frequency learning loop; the high-frequency control loop has a cycle of no less than 100 Hz with the frame rate of the image sensing module as the period, and performs the whole process from image acquisition, feature dynamic extraction, multi-modal motion inference to instruction analysis and drive output; The low-frequency learning loop is triggered at a fixed time interval or when the tracking performance index is lower than a threshold, and has a cycle of no less than 1 second, and performs the parameter update of the policy network and the value network in the adaptive policy generation unit, and the adjustment of the neural network in the feature dynamic extraction unit.

Images