Intelligent automated multi-micro-operational device cell surgery method and system

Abstract

This invention discloses an intelligent automated cell surgery method and system using multiple micromanipulation devices, belonging to the fields of biomedical engineering and automation control. The core of this method lies in the independent workspace partitioning and temporal concurrent operation scheduling strategy for multiple micromanipulation devices. By establishing a global coordinate system and generating a gridded digital map, images are acquired during traversal of each region, and an AI model is invoked to identify target cells and the positions of the micromanipulation device ends. Based on the spatial distribution of the devices, independent workspaces are partitioned for each device, ensuring no interference. Under the premise of exclusive management of camera access rights, multiple devices are driven to perform surgical operations concurrently in a temporal sequence. After completing the task in the current region, the devices and cameras are moved to the next region according to an anti-collision strategy until traversal is complete. The above process is automatically repeated according to task requirements until all target cells have been identified and operated on. This invention achieves high-throughput, high-precision, and low-interference collaborative operation of multiple micromanipulation devices, significantly improving the efficiency and success rate of cell surgery.

Description

Technical Field

[0001] This invention belongs to the fields of artificial intelligence, biomedical engineering and automation control technology, and relates to an intelligent automated cell surgery method and system with multiple micromanipulation devices. It particularly emphasizes the division of independent working space and the time-sequential concurrent operation scheduling of multiple micromanipulation devices. Background Technology

[0002] In fields such as neuroscience, cell biology, and drug development, precise micromanipulation of ex vivo tissue sections or live cell samples, such as single-cell injection, electrophysiological signal recording, and immunohistochemical staining, is a key technique for elucidating the mechanisms of complex life activities. Currently, existing micromanipulation techniques mainly rely on manual or semi-automatic / fully automated single-micromanipulation device systems. In manual operation, experienced researchers manually control individual micromanipulation devices (such as micropipettes or electrodes connected to a micro-manipulator) to locate and process target cells one by one, relying on visual feedback from the microscope camera—a time-consuming and labor-intensive process. Semi-automated systems, with the addition of basic image recognition capabilities, can assist operators in locating cells requiring specific labeling more quickly and accurately. This can, to some extent, meet the needs of small-scale, low-throughput experiments. While fully automated single-micromanipulation systems integrating intelligent algorithms can automate target cell localization and single-manipulation device operations, the serial operation of these devices cannot fully utilize the idle time of the microscope camera during cell surgery, making it difficult to meet the high-throughput, multi-parameter analysis requirements of modern biomedical research.

[0003] Therefore, a smart, multi-manipulation device collaborative parallel cell surgery method is needed to efficiently utilize microscopic resources and address the contradiction between low throughput, single parameters, and the demands of modern experiments caused by the inefficiency of serial operations. However, multi-manipulation device collaboration faces significant spatial interference risks. When attempting to introduce multiple micromanipulation devices within a limited microscopic field of view and physical space to increase throughput, the lack of effective work area division and corresponding control device scheduling makes it highly susceptible to mechanical collisions or interference of movement paths at the ends of the micromanipulation devices, posing a significant risk of damage to expensive cell samples and delicate micromanipulation devices. Furthermore, existing systems cannot achieve dynamic scheduling of shared resources such as microscope cameras. When multiple micromanipulation devices simultaneously request visual feedback, system resource deadlock is highly likely, making it impossible to truly achieve efficient and safe parallel operation of multiple micromanipulation devices in both time and space. Summary of the Invention

[0004] To address the low throughput problem of cell-level manipulation in existing technologies, this invention proposes an intelligent automated multi-micromanipulation device cell surgery method and system. Through two core innovations—independent job space partitioning based on the number of micromanipulation devices and initial angle pose, and time-division multiplexing of single micromanipulation devices for concurrent operation scheduling—this invention can fully utilize the idle time of microscope camera resources and effectively solve the technical problem of easy interference in multi-device collaboration in parallel cell surgery methods, thereby achieving high-throughput, multi-job cell-level manipulation.

[0005] To achieve the above objectives, this invention provides an intelligent automated multi-manipulation device cell surgery method, including a fully automated cell surgery process encompassing digital map generation, independent workspace partitioning for micromanipulation devices, and concurrent execution in a time sequence, as well as a complete software and hardware system for implementing the aforementioned process; the automated process specifically comprises: First, a unified global coordinate system is established to eliminate distance and angle errors between the independent coordinate systems of each micromanipulation device and the microscope camera coordinate system. On this basis, a standard gridded digital map is adaptively generated for the user-defined task area according to the actual microscope field of view. The rows and columns of the grid corresponding to each microscope field of view are strictly perpendicular, and the camera movement path is planned according to any scanning strategy that covers the entire sample, such as serpentine scanning. Subsequently, the scanning path sequentially traverses each microscope field of view in the digital map. Within each field of view, based on the initial angle and spatial distribution of each micromanipulation device, and according to the optimal reachable area that does not intersect, multiple microscope fields of view adjacent to the current position are assigned independent and physically independent working spaces for each micromanipulation device. Then, under the premise of exclusive management of microscope camera access rights, multiple micromanipulation devices are driven to concurrently perform cell surgery actions in their respective independent working spaces according to the time-series scheduling. During this process, an artificial intelligence model is invoked in real time to locate the end effector of the micromanipulation device, and the pose of the micromanipulation device is dynamically corrected based on the positioning results to compensate for positional offset. After one round of concurrent tasks is completed, the manipulation device moves to the next path point in the scanning path according to the anti-collision area switching mechanism to start the next round of tasks. This cycle continues until all task areas have been traversed.

[0006] Preferably, the process of establishing a global coordinate system includes: The physical properties of each micromanipulation device and microscope are read, and coordinate transformation and normalization calibration are performed on the angular error between the independent coordinate system of each micromanipulation device and the camera coordinate system. The coordinate system of all micromanipulation devices is aligned with the physical coordinate system of the microscope camera to ensure the consistency of subsequent spatial planning and motion control.

[0007] Preferably, generating a digital map specifically involves: The user actively selects multiple coordinate points in the global coordinate system and defines the task area based on the envelope of the coordinate points; the task area can be of any shape. Within the user-selected polygonal task area, a compactly arranged viewpoint sampling orthogonal mesh is constructed based on the camera's field of view size. Adaptive tiling is achieved through boundary clipping and hole masking to ensure that the mesh covers the entire area without omission and to automatically exclude non-target cavities within the area.

[0008] The camera movement path points connecting the centers of each grid are generated according to the sample scanning order, which serve as the scanning path; wherein, the starting direction of the sample scanning is adaptively determined according to the placement direction of the digital map. When the digital map is placed horizontally, it first scans along the rows and then moves vertically. When the digital map is placed vertically, it first scans along the columns and then moves horizontally. Preferably, the process of dividing and assigning cell surgery tasks within the fields of view of multiple adjacent cameras to corresponding micromanipulation devices includes: First, obtain the spatial position and initial angle of all micromanipulation devices; Initialize the starting position of the micromanipulation devices. Based on the number of micromanipulation devices, assign each device to an adjacent microscopic field-of-view grid corresponding to the starting point of the digital map. Within multiple adjacent microscopic fields of view, assign a corresponding micromanipulation device to each field of view according to angular relationships and the principle of non-intersection, as its independent working space. Within this independent working space, the connecting rods of the micromanipulation devices are rays extending from the coordinates of the currently assigned micromanipulation device in the direction of the micromanipulation device's angle. These rays strictly adhere to the principle of non-intersection and will not interfere with other micromanipulation devices or their corresponding working spaces. Based on the movement direction of the serpentine path shared by each micromanipulator, the system determines the task order of each micromanipulator according to the principle of prioritizing the execution of micromanipulators on the same side of the serpentine path direction. The system also divides the task sequence and workspace pairing set according to the number of micromanipulators. These task sequences are executed sequentially based on the order. During each operation, the current micromanipulator is located in an independent workspace that is adjacent to but does not interfere with other devices, thereby achieving physical isolation in space. Preferably, the process of exclusively managing access to the microscope camera includes: A dynamic locking and unlocking mechanism for microscope camera resources is established. When any micromanipulation device needs to perform visual positioning or calibration, it requests control of the microscope camera from the system. If the camera is unlocked at this time, the visual task of the micromanipulation device is executed; otherwise, the request is added to the microscope camera request priority list, and scheduled according to the priority order of the list when the camera is unlocked. Based on the above strategy, it is ensured that only one micromanipulation device holds control of the microscope camera at any given time. After any manipulation device completes a visual-related task or detects no target in its field of view, it immediately releases control of the microscope camera so that the system can schedule other micromanipulation devices to take over, thereby maximizing the utilization rate of microscope camera resources.

[0009] Preferably, the external device includes a current device or a fluid control device; after performing the cell surgery operation, when the next task arrives, the light source is switched and images are acquired to determine the result of the previous cell surgery, and then the execution status is returned.

[0010] Preferably, after performing cell surgery, the micromanipulation device enters a cell reaction waiting state until the preset reaction time ends; during the waiting period, the micromanipulation device immediately releases camera control, and the system then schedules other micromanipulation devices to continue working according to the microscope camera request priority list, making full use of the microscope camera's idle time during the cell reaction waiting period, and realizing the close alternation and concurrency of multiple micromanipulation devices on the time axis.

[0011] Preferably, before the micromanipulation device moves to the next workspace according to its planned path, the microscope camera is first raised to a safe height, then the micromanipulation device is moved, and then the microscope camera is moved and its coordinates are recalibrated to avoid mechanical collisions between the microscope camera and the micromanipulation device.

[0012] Preferably, the complete software and hardware system for realizing the above-mentioned automated cell surgery process includes: Image processing and device guidance module: used for target detection and segmentation of microscopic images, identification of target cells and the ends of micromanipulation devices, and guidance of the movement of micromanipulation devices and camera; Multi-micro-operation device collaborative control module: used for global modeling, map planning, task area division, and management of mutual exclusion access and concurrent scheduling of multiple micro-operation devices to shared resources based on state machine and priority queue; Hardware execution layer: includes microscope, camera, multiple micro-manipulation devices, displacement stage and external devices, used to execute physical actions in response to the instructions of the collaborative control module.

[0013] Compared with the prior art, the present invention has the following advantages and technical effects: This invention overcomes the bottleneck of cell reaction waiting time in serial operation of a single micromanipulation device by introducing a time-sequential concurrent operation mechanism based on a state queue. After cell procedures such as injection staining, micromanipulation devices require a necessary physical diffusion or chemical reaction waiting period. This system automatically captures this time gap, fully utilizing the idle time of the microscope camera during serial operation, and scheduling multiple micromanipulation devices to take over the microscope camera and execute actions. This "time-division multiplexing" strategy enables multiple micromanipulation devices to achieve tight alternation and concurrency on the time axis, significantly reducing redundant waiting time and resulting in a multiple-fold increase in overall experimental throughput.

[0014] This invention employs a rigorous global spatial planning and local gridded allocation strategy to effectively address the physical interference problem that easily arises when multiple micromanipulation devices operate collaboratively. Based on the physical angles and spatial distribution of the multiple micromanipulation devices, the system adaptively divides adjacent microscope fields of view into a corresponding number of independent, non-overlapping regions, allocating a dedicated, independent working space for each micromanipulation device. Each device is only responsible for and allowed to process cell targets within its assigned camera field of view, fundamentally achieving physical isolation of movement paths and completely eliminating the risk of cross-interference or mechanical collisions between multiple devices at the same location, ensuring the high safety of the concurrent multi-micromanipulation device system.

[0015] This collaborative control strategy of "global planning-local parallelism-regional advancement" enables the system to simultaneously process large-scale, irregular biological samples at high throughput, significantly shortening the overall cell surgery time and greatly increasing experimental throughput.

[0016] This invention achieves complete automation of the entire process—from digital map generation, scanning path planning, target recognition, task allocation, concurrent execution to area switching—without human intervention. This solution reduces reliance on operator experience, eliminates the uncertainties of human operation, and ensures a high degree of repeatability and standardization in cell surgery procedures, making it suitable for large-scale, multi-batch biomedical experiments.

[0017] The adaptive map generation technology of this invention allows users to define task areas of arbitrary shapes according to experimental needs, and can automatically perform grid segmentation based on the camera field of view size and the number of micro-manipulation devices. This enables the system to flexibly adapt to biological tissue samples of different sizes and shapes (such as brain slices, organoids, etc.) and different numbers of micro-manipulation devices, and has wide applicability. Attached Figure Description

[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1This is a diagram illustrating the overall architecture of the intelligent automated multi-micromanipulation device cell surgery system according to an embodiment of the present invention. Figure 2 This is a flowchart of the intelligent automated multi-micromanipulation device cell surgery method according to an embodiment of the present invention; Figure 3 This is a timing diagram of the concurrent operation of multiple micro-operation devices according to an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the generation of a digital map according to an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the adaptive serpentine path generation based on map shape, according to an embodiment of the present invention. Detailed Implementation

[0019] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0021] Example 1 like Figure 1 As shown, the intelligent automated cell surgery system adopts a layered architecture design, divided from bottom to top into a hardware layer, an algorithm layer, an interaction layer, and a data layer. Each layer works together to support the collaborative operation of multiple micromanipulation devices. The following is a description of each layer and its core modules: (1) The hardware layer (basic support) is located at the lowest level of the system, providing physical execution and sensing capabilities. It mainly includes: Multi-operation device (robotic arm): responsible for performing micro-operations such as puncture, injection, and cutting in cell surgery.

[0022] Microscope: Used to acquire microscopic images of cells and micromanipulated devices in real time, providing a data source for visual servoing.

[0023] Light source equipment: Provides a stable and adjustable lighting environment for microscope imaging, ensuring image quality.

[0024] Other external devices (such as current devices): used for auxiliary function modules such as electrical stimulation and impedance detection.

[0025] (2) The algorithm layer (core intelligent engine) contains the key algorithm modules required for system operation and is divided into three main sub-functions: Spatial planning: including global coordinate transformation, independent workspace division, serpentine path point generation and spatial interference verification, is used to plan the motion trajectory and work area of ​​multiple devices to avoid collisions.

[0026] Temporal parallelism: Through state machine-driven, load balancing and time-sharing strategies, concurrent scheduling and efficient resource utilization of multiple devices in the time dimension are achieved.

[0027] Image processing includes image preprocessing, gradient algorithms (for autofocus), feature extraction, and several deep learning models (pinpoint recognition, cell segmentation, cell classification, and result discrimination) for target detection, localization, and surgical outcome evaluation.

[0028] (3) Interaction Layer (System Management and Control Center): This layer provides visualization and control interfaces for operators and internal system management. Map Management: Manages the global gridded digital map and maintains the regional traversal status.

[0029] Operator management: Monitor and control the status, permissions, and task assignments of each robotic arm.

[0030] Hardware communication: Responsible for issuing instructions and exchanging data with various devices in the hardware layer.

[0031] Perfusion information management: Record and manage parameters such as flow rate and duration during cell culture medium perfusion.

[0032] Visual interface: Provides a graphical operation panel that displays images, maps, device status, and task progress in real time.

[0033] (4) The data layer (information storage and status recording) is responsible for data persistence and retrieval during system operation: Organizational information: Stores basic attributes and organizational structure data of experimental samples.

[0034] Global coordinates: Records the precise coordinates of each device, target cell, and region on the map.

[0035] Cell list: Maintains the number, category, location, and surgical status of identified target cells.

[0036] Perfusion results: Save the parameters and results data for each perfusion operation.

[0037] Task Status: Tracks the current traversal progress, the execution stage of each device, and the overall task completion status.

[0038] In this embodiment, the data layer provides the algorithm layer with inputs such as coordinates and cell information. The algorithm layer calculates and generates control commands, which are then sent to the hardware layer for execution via the interaction layer. Simultaneously, the interaction layer acquires image and status data from the hardware layer and feeds it back to the algorithm layer for closed-loop control, ultimately presenting the data to the user through a visual interface. The overall design achieves a closed-loop automated process of "perception → decision → execution → recording".

[0039] like Figure 2 As shown in the example, this application demonstrates the intelligent automated multi-micromanipulation device cell surgery method of the present invention by using four micromanipulation devices to simultaneously perform a cell perfusion task. The four micromanipulation devices are four perfusion needles. A microscope camera, current device, light source device, and displacement stage are connected to the four perfusion needles and are uniformly scheduled by a collaborative control module. The sample information to be processed can be pre-entered into the system, and this information includes at least the target cell state, perfusion-related parameters, and the target cell's position coordinates in the global coordinate system.

[0040] In this application example, the specific implementation process includes the following steps: 1. Power on the infusion hardware and software: First, turn on the microscope camera control switch, the control switches for the four infusion needles, the microscope camera imaging switch, the light source switch, and the current control switch, respectively. Then, start the software system. The software will automatically read all the above device information and status, and automatically update and display the status of each hardware device within the software. If the device is not powered on or if there are interface changes, the user should make adjustments according to the specific situation. 2. System Initialization: First, the operating environment model is loaded, establishing communication between the application logic layer, hardware device layer, and data management layer. The system obtains the initial physical parameters (such as length, position, initial angle, and microscope camera field of view) of the four perfusion needles and microscope camera by reading the configuration file, and initializes the motion control communication interface between the microscope camera and multiple devices. Simultaneously, the system automatically detects and loads the artificial intelligence models required for visual reasoning, such as YOLO-based cell detection models, cell segmentation models, needle tip detection models, and perfusion result classification models. Through these initialization steps, the system establishes communication at both the hardware and software levels, ensuring the effective execution of subsequent perfusion procedures. 3. Establish a unified coordinate system: Map the motion spaces of the four micromanipulation devices, each operating independently in its own coordinate system, to the visual coordinate system of the microscope camera, completing the normalization calibration. Considering that each needle and the microscope camera have independent coordinate systems—for example, the coordinate system of the micromanipulation device represents oblique radial or tangential movement, and there is an angular error between it and the orthogonal coordinate system of the camera—the system calculates the distance error between the center of each needle's coordinate system and the center of the camera's coordinate system, and calculates the corresponding affine transformation matrix based on the error between the angle of each needle and the orthogonal coordinate system of the camera, thus unifying all needles and the camera to the same physical coordinate system. 4. Digital Map Generation of the Perfusion Area: A polygonal task area is generated based on the user-selected cell task area. Combining the camera's field of view and the spatial arrangement of the four perfusion needles, the task area is divided into multiple gridded sub-task areas, ensuring that each sub-task area matches the physical distribution of the four perfusion needles. A scanning path connecting the centers of each grid is then generated according to the sample scanning method, serving as the basis for the movement of the microscope camera and related actuators traversing each sub-task area.

[0041] Specifically, such as Figure 4 As shown, when generating a digital map, the user-selected area is cropped and polygonally constrained. A tight spatial arrangement of orthogonal grids is calculated based on the camera's field of view, and the polygon map is adaptively meshed to ensure that all locations within the selected task area are covered by the camera's field of view, while avoiding areas such as cavities within the user's envelope. Figure 5 As shown, the adaptive generation process of sample scanning path points is as follows: when the map is placed horizontally, it first scans along the rows and then moves vertically; when the map is placed vertically, it first scans along the columns and then moves horizontally. 5. Independent Workspace Division: Based on the spatial distribution of target cells within the current area, the positional relationship of the four perfusion needles relative to the camera, and the reachability of each needle, the system divides the cell perfusion tasks within the current area and assigns them to the corresponding perfusion needles, defining an independent and physically independent workspace for each needle. For adjacent camera fields of view already assigned to a corresponding perfusion needle, the system further generates the execution order of the four perfusion needles within the current sub-task area according to the spatial orientation of the four needles and the overall serpentine path movement direction, prioritizing needles on the same side. This ensures that each perfusion needle sequentially completes the corresponding cell perfusion task within its own workspace.

[0042] Furthermore, in this example, the four perfusions are located at the four corners of the biological tissue. The system automatically divides the adjacent field of view into four 2×2 physically isolated quadrants and maps these four fields of view to the needles at the corresponding four corners. Each field of view is regarded as an independent working space for a needle. Through this strict decoupling of physical space, it is ensured that when each device moves and operates within its own independent grid, it will not cross the boundary and cause mechanical collisions, nor will it cause visual target contention and path interference.

[0043] 6. Perform cell surgeries concurrently in a sequential manner. For example... Figure 3 As shown in the timing diagram of concurrent operation of multiple micromanipulation devices, under the premise of exclusive management of microscope camera access, the system drives four perfusion needles to alternately and concurrently perform cell perfusion operations in their respective independent workspaces according to the execution order generated in step 5 through timing scheduling. For any scheduled perfusion needle, its single-cell perfusion process includes: (1) Move the perfusion needle to the corresponding position above the target cells; (2) Perform automatic calibration of the infusion needle to correct the positional error of the needle tip in the current visual coordinate system; (3) Perform image acquisition and target cell search near the target cell to determine the center of the target cell and the optimal insertion position of the perfusion needle (e.g., twice the radius of the cell). (4) Once the micromanipulation device and the cell are in a suitable position for the operation, the system switches the micromanipulation device to low speed mode to prevent the micromanipulation device from damaging the cell and causing failure. The system controls the perfusion needle to advance the needle in the radial direction and insert it into the target cell. During the approach, the system adjusts the relative position of the needle tip and the target cell in combination with image feedback. (5) When the perfusion needle reaches the set perfusion position, the current device is activated so that the dye is injected into the target cell under the action of the current; at the same time, the perfusion needle enters the diffusion reaction waiting period to ensure that the dye has enough time to diffuse to all synaptic branches of the cell (such as Lucifer Yellow dye diffusion requires 5 minutes).

[0044] While one perfusion needle is executing the above process, other perfusion needles enter waiting, ready, or executing states according to the timing schedule. When camera access is released, the next perfusion needle that meets the conditions obtains camera access and continues execution, thus enabling the four perfusion needles to run concurrently in a timed alternation within the same system. For cells that have already been perfused within the current subtask region, the system updates their task status to avoid duplicate execution.

[0045] 7. Perfusion Result Judgment and Anti-collision Area Switching. Once all perfusion tasks assigned to the four perfusion needles within the current 2x2 field of view have been executed and entered the reaction waiting period, the needle on the same side as the overall serpentine path direction, as described in step 5, is executed first, thus ending its waiting period first. After the first needle finishes its waiting period, the camera is captured, the cell perfusion result is judged, and the result is entered into the system. Then, this needle moves to the target area of ​​the next 2x2 field of view before the remaining three needles, and performs anti-collision area switching. The specific steps are as follows: The system raises the microscope camera to a safe height, controls the current needle to move to the center of its assigned independent working space within the next 2x2 field of view, then moves the camera to the needle's position and lowers it to surface height, performing needle tip calibration to eliminate coordinate errors caused by needle movement. After confirming that it has moved to the target position, the system performs the cell surgery task in step 6 for that needle, and releases the camera after entering the waiting period. The same anti-collision area switching is performed on the remaining 3 needles until all 3 needles have moved into the next 2x2 field of view. The above "switching area - performing task - waiting" steps are performed alternately and concurrently in this order until all independent working areas have been traversed.

[0046] Based on the execution order in step 5, the needle that is given priority for infusion moves in the same direction as the overall microscope camera in a serpentine motion. Therefore, when it moves first, there are no other needles in front of it, avoiding the risk of collision. At the same time, after it finishes moving, it makes enough safe space for other needles to follow, ensuring the safe movement of the remaining needles.

[0047] 8. As needed for the task, the system can automatically restore to the initial pose of the overall task and continue scanning other cells within the field of view, repeating steps 4-7 until all target cells in the task area have been identified and perfused. Through the above implementation method, this embodiment incorporates the complete automated perfusion process of 4 perfusion needles into a unified global coordinate, digital map generation, independent work space division, temporal concurrent scheduling, and anti-collision area switching framework, realizing the automatic collaborative execution of 4-needle perfusion tasks. The concurrent perfusion of 4 needles greatly improves the throughput of cell surgery and the overall work speed, and provides real-time safety monitoring under real-time calibration and visual correction.

[0048] The above implementation process is only a specific description of the steps of the present invention, and the order of steps and functional boundaries are consistent with the technical solutions disclosed in the present invention.

[0049] The above is merely one specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A smart, automated cell surgery method using multiple micromanipulation devices, characterized in that, Includes the following steps: Digital map generation: Establish a unified global coordinate system and generate a gridded digital map based on the user-defined task area. The digital map contains multiple sub-task areas and their corresponding scanning paths. Independent workspace division: During the process of traversing each sub-task area according to the scanning path, based on the spatial distribution of each micromanipulation device, the cell surgery task in the current area is divided and assigned to the corresponding micromanipulation device, and an independent and physically non-interfering workspace is defined for each micromanipulation device. Concurrent execution of cell surgery in a time sequence: Under the premise of exclusive management of microscope camera access rights by a single manipulator, multiple micromanipulators are driven to perform cell surgery actions concurrently in their respective independent workspaces through time sequence scheduling; Anti-collision area traversal: After completing all tasks within the current sub-task area, control the micro-manipulation device and microscope camera to move to the next sub-task area in the digital map until all areas have been traversed; Based on the task requirements, the system rescans the entire area from the initial position and performs the above steps until all cells are identified and the surgery is performed.

2. The method according to claim 1, characterized in that, The process of establishing a global coordinate system includes: reading the initial physical parameters of each micromanipulation device and the microscope camera; performing coordinate transformation for the angular error between the independent coordinate system of each micromanipulation device and the camera coordinate system; and mapping the independent motion space of each micromanipulation device to the visual coordinate system of the microscope camera to achieve normalization calibration of the heterogeneous coordinate system.

3. The method according to claim 1, characterized in that, The specific steps for generating a digital map are as follows: Generates a polygonal task region of arbitrary shape based on the envelope range of the coordinate points selected by the user; Based on the camera's field of view and the number of micro-manipulation devices, the task area is adaptively divided into multiple spatially arranged grid cells, with each grid corresponding to a sub-task area, ensuring that the spatial distribution of each sub-task area matches the physical arrangement of the multiple micro-manipulation devices. The camera movement path points connecting the centers of each grid are generated according to the sample scanning order, which serve as the scanning path; wherein, the traversal order of the sample scanning path points is adaptively adjusted according to the aspect ratio and arrangement direction of the grid.

4. The method according to claim 1, characterized in that, The process of dividing and assigning cellular surgical tasks within the current area to corresponding micromanipulation devices includes: Based on the initial angle and pose of the micromanipulator, determine the task execution priority and initial response region of each micromanipulator; The adjacent microscope fields of view corresponding to the number of micromanipulation devices are mapped to the work space according to the principle that the motion trajectory does not intersect, so as to achieve physical isolation of the work space of multiple micromanipulation devices. When performing a task, the target cells identified in the current field of view are directly assigned to the corresponding micromanipulation device, ensuring that the independent working space of each micromanipulation device is the same as the field of view of a microscope camera, thus avoiding cross-interference of multiple devices on the same target.

5. The method according to claim 1, characterized in that, The process of exclusively managing access to a microscope camera includes: Establish a dynamic locking and unlocking mechanism for camera resources; maintain a state queue containing the current task status for each micro-operation device; When any micromanipulation device needs to perform visual positioning or calibration, it requests camera control. The system performs timing scheduling based on the state queue and ensures that only one micromanipulation device holds camera control at any given time. After completing a vision-related task or detecting no target within its field of view, the micro-manipulation device immediately releases camera control, enabling devices in a waiting or idle state to promptly relinquish visual resources, thus achieving time-division multiplexing and concurrent operation of multiple devices.

6. The method according to claim 1, characterized in that, The surgical procedure includes a timing control strategy: when a specific micromanipulation device enters a preset waiting state (such as a reaction process like dye diffusion), the system utilizes time gaps to automatically schedule other micromanipulation devices requesting camera control to continue operating through a state queue, maximizing the efficiency of concurrent operation of multiple devices in a timing alternation manner.

7. The method according to claim 1, characterized in that, The area switching includes collision avoidance logic: before moving to the next sub-task area, the camera is first raised to a safe height, and then the micro-operation devices located in front of the path are moved first according to the direction of movement. Then the camera and the remaining micro-operation devices are moved. Combined with the designated independent working space, mechanical collisions and mutual interference during the spatial movement of multiple devices are avoided to the greatest extent.

8. A smart automated multi-micromanipulation device cell surgery system for implementing the method of any one of claims 1-7, characterized in that, include: Image processing and device guidance module: used for target detection and segmentation of microscopic images, identification of target cells and the ends of micromanipulation devices, and guidance of the movement of micromanipulation devices and camera; Multi-micro-operation device collaborative control module: used for global modeling, map planning, task area division, and management of mutual exclusion access and concurrent scheduling of multiple micro-operation devices to shared resources based on state machine and priority queue; Hardware execution layer: includes microscope, camera, multiple micro-manipulation devices, displacement stage and external devices, used to execute physical actions in response to the instructions of the collaborative control module.

Images