Sample plate transfer control method and system for intelligent laboratory and related device

Abstract

The invention provides a sample plate transfer control method and system for an intelligent laboratory and a related device. The method comprises the following steps: controlling a humanoid robot to move to a first target position based on initialization task information; determining, from the environmental awareness and navigation system, a capture error compensation value based on the first sensing data; the humanoid robot is controlled to conduct material grabbing operation based on the grabbing error compensation value, and fine adjustment is conducted based on the first feedback signal so that the humanoid robot can complete the material grabbing operation; the material grabbing state is detected, if the grabbing state is completed, the humanoid robot is controlled to move to a second target position, and the state is dynamically adjusted; and when it is detected that the humanoid robot reaches the second target position, the target sample plate is placed based on third feedback information. Thus, through scene compliance control and vision-force sense cooperative control, the efficiency and adaptability of the mechanical arm in laboratory automation are improved, and the sample transfer reliability and efficiency in a high-throughput experiment scene are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot control, and in particular to a sample plate transfer control method and system for an intelligent laboratory and related devices. BACKGROUND

[0002] Intelligent laboratories have higher and higher requirements for automation, and the task process of an experimental task may involve devices of various types, functions and models. Therefore, a mechanical arm is used to transport materials between different devices, for example, to perform a sample plate transport operation. Many experimental devices are currently provided with multiple sample plate storage positions, and the plate station for storing sample plates generally has multiple plate positions. During one transport process, a humanoid robot may need to take a sample plate from a starting position of a first plate station of a device and then transfer the sample plate to a target position of a second plate station on another device.

[0003] However, the current sample plate transfer scheme based on a mechanical arm has the problems of insufficient simplification of the experimental system for building an experimental task, insufficient precision of the sample plate grasping, low efficiency of the sample plate transfer task in the experimental task due to the constraints of the mechanical arm grasping and motion capabilities, and poor adaptability and reliability. SUMMARY

[0004] The embodiments of the present application provide a sample plate transfer control method and system for an intelligent laboratory and related devices, which can improve the sample plate transfer adaptability, and improve the sample transfer reliability and efficiency in a high-throughput experimental scenario.

[0005] In a first aspect, the embodiments of the present application provide a sample plate transfer control method for an intelligent laboratory, applied to a controller of a humanoid robot system, wherein the humanoid robot system further includes the humanoid robot, the humanoid robot includes a mechanical arm, the mechanical arm is provided with a gripping device, the gripping device is used to grasp a material, the material includes a target sample plate, the humanoid robot system further includes a plate station module and an environment perception and navigation system, and the plate station module includes a first target position and a second target position; the method includes:

[0006] obtaining task information and determining initialization task information based on the task information, and controlling the humanoid robot to move from an initial position to the first target position based on the initialization task information;

[0007] obtaining first sensing data of the humanoid robot at the first target position from the environment perception and navigation system, and determining a grasping error compensation value based on the first sensing data, wherein the first sensing data includes first visual data and first mechanical data;

[0008] control the humanoid robot to perform a material grabbing operation on the target sample plate based on the grabbing error compensation value, and obtain a first feedback signal, and based on the first feedback signal, fine-tune the humanoid robot to complete the material grabbing operation, the first feedback signal being second visual data and / or second mechanical data detected during the material grabbing operation;

[0009] detect a material grabbing state, and if it is detected that the material grabbing state is a completed grabbing state, control the humanoid robot to move to a second target position based on the initial task information, and dynamically adjust the state during the movement based on second feedback information, the second feedback information including third visual data and / or third mechanical data detected during the movement;

[0010] when it is detected that the humanoid robot reaches the second target position, obtain third feedback information and place the target sample plate based on the third feedback information, the third feedback information including fourth visual data and / or fourth mechanical data.

[0011] In a second aspect, embodiments of the present application provide a humanoid robot system applying a sample plate transfer control method of an intelligent laboratory as described in the first aspect, comprising:

[0012] a humanoid robot, the humanoid robot comprising a mobile chassis and a mechanical arm, the mechanical arm being provided with a grabbing device, the grabbing device being used to grab a material, the material including the target sample plate, the mobile chassis comprising a biped structure and / or a wheeled structure, the wheeled structure being switched when the humanoid robot is moving, and the biped structure being switched when the humanoid robot is standing;

[0013] a controller, configured to execute the method as described in the first aspect above;

[0014] an environment perception and navigation system, the environment perception and navigation system comprising a visual system, a six-dimensional force sensor, and a navigation positioning system, the visual system being configured to obtain visual data, the six-dimensional force sensor being configured to obtain mechanical data, and the navigation positioning system being configured to provide path information and space information of the intelligent laboratory;

[0015] a plate station module, the plate station module being a plurality of modular areas for storing sample plates, the plate station module being a multi-modular design.

[0016] In a possible embodiment, the visual system comprises a plurality of cameras, the plurality of cameras including a head camera and a mechanical arm camera;

[0017] The six-dimensional force sensor comprises a mechanical arm mechanical sensor for sensing the force condition of the grabbing device and / or the end of the mechanical arm; the plate station is a multi-layer plate station built by a C-shaped channel steel frame, and the plate station comprises an operation area, and each operation area is provided with at least one positioning code.

[0018] The controller, the head camera, the mechanical arm camera, and the six-dimensional force sensor are time-space synchronous.

[0019] The mechanical arm of the humanoid robot comprises a mechanical arm with 7 degrees of freedom.

[0020] In a third aspect, the embodiments of the present application provide a sample plate transfer control device of an intelligent laboratory, applied to a controller of a humanoid robot system, wherein the humanoid robot system further comprises the humanoid robot, the humanoid robot comprises a mechanical arm, the mechanical arm is provided with a grabbing device, the grabbing device is used for grabbing materials, the materials comprise target sample plates, the humanoid robot system further comprises a plate station module and an environment perception and navigation system, and the plate station module comprises a first target position and a second target position; the device comprises:

[0021] An initialization module is configured to acquire task information, determine initialization task information based on the task information, and control the humanoid robot to move from an initial position to the first target position based on the initialization task information.

[0022] A first material grabbing module is configured to acquire first sensing data of the humanoid robot at the first target position from the environment perception and navigation system, determine a grabbing error compensation value based on the first sensing data, and acquire first feedback signals based on the grabbing error compensation value.

[0023] A second material grabbing module is configured to control the humanoid robot to perform a material grabbing operation on the target sample plates based on the grabbing error compensation value, and perform fine tuning based on the first feedback signals so that the humanoid robot completes the material grabbing operation, wherein the first feedback signals are second visual data and / or second mechanical data detected during the material grabbing operation.

[0024] A material transfer module is configured to detect a material grabbing state, and if it is detected that the material grabbing state is a completed grabbing state, control the humanoid robot to move to the second target position based on the initialization task information, and perform state dynamic adjustment based on second feedback information during the movement, wherein the second feedback information comprises third visual data and / or third mechanical data detected during the movement.

[0025] The material placing module is configured to, when it is detected that the humanoid robot reaches the second target position, acquire third feedback information and place the target sample plate based on the third feedback information, the third feedback information including fourth visual data and / or fourth mechanical data.

[0026] In a fourth aspect, an electronic device is provided, which includes a processor, a memory, a communication interface, and one or more programs. The one or more programs are stored in the memory and configured to be executed by the processor. The programs include instructions for performing some or all of the steps described in the first aspect.

[0027] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program. The computer program includes program instructions, which, when executed by a processor, cause the processor to perform some or all of the steps described in the first aspect.

[0028] In a sixth aspect, a computer program product is provided, which includes a non-transitory computer-readable storage medium storing a computer program. The computer program is operable to cause a computer to perform some or all of the steps described in the first aspect. The computer program product can be a software installation package.

[0029] By implementing the embodiments of the present application, the task information is acquired, the initialization task information is determined based on the task information, and the humanoid robot is controlled to move from an initial position to a first target position based on the initialization task information; the first sensing data of the humanoid robot at the first target position is acquired from the environment perception and navigation system, and a grasping error compensation value is determined based on the first sensing data, the first sensing data including first visual data and first mechanical data; the humanoid robot is controlled to perform a material grasping operation on a target sample plate based on the grasping error compensation value, and a first feedback signal is acquired, and fine tuning is performed based on the first feedback signal so that the humanoid robot completes the material grasping operation, the first feedback signal being second visual data and / or second mechanical data detected during the material grasping operation; a material grasping state is detected, and if it is detected that the material grasping state is a completed grasping state, the humanoid robot is controlled to move to a second target position based on the initialization task information, and state dynamic adjustment is performed based on second feedback information during the movement, the second feedback information including third visual data and / or third mechanical data detected during the movement; when it is detected that the humanoid robot reaches the second target position, third feedback information is acquired and the target sample plate is placed based on the third feedback information, the third feedback information including fourth visual data and / or fourth mechanical data. In this way, through scenario-based compliant control and visual-haptic collaborative control, the efficiency and adaptability of the robotic arm in laboratory automation are improved, and the sample transfer reliability and efficiency in a high-throughput experiment scenario are significantly improved. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background art, the drawings needed to be used in the embodiments of the present application or the background art will be described below.

[0031] Figure 1a is a schematic diagram of an architecture of a humanoid robot system provided by an embodiment of the present application;

[0032] Figure 1b is a schematic diagram of another architecture of a humanoid robot system provided by an embodiment of the present application;

[0033] Figure 1c is a schematic diagram of a structure of a humanoid robot provided by an embodiment of the present application;

[0034] Figure 2 is a flowchart of a sample plate transfer control method of an intelligent laboratory provided by an embodiment of the present application;

[0035] Figure 3 is another flowchart of a sample plate transfer control method of an intelligent laboratory provided by an embodiment of the present application;

[0036] Figure 4 is a structural schematic diagram of a sample plate transfer control device of an intelligent laboratory according to an embodiment of the present application;

[0037] Figure 5 is a structural schematic diagram of another sample plate transfer control device of an intelligent laboratory according to an embodiment of the present application;

[0038] Figure 6 is a structural schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative work fall within the scope of protection of the present application.

[0040] The terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish different objects, and are not used to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or electronic device that includes a series of steps or units is not limited to the listed steps or units, but in an alternative example, also includes steps or units not listed, or in an alternative example, also includes other steps or units inherent to the process, method, product, or electronic device.

[0041] In this document, the term "embodiment" means that the specific features, structures, or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment is referred to, nor does it mean that other embodiments are mutually exclusive or alternative to the one described. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0042] The current sample plate transfer scheme based on a mechanical arm has the problems that the experimental system for an experimental task is not simple enough, the precision of grabbing a sample plate is not high enough, the transfer task of the sample plate in the experimental task is also constrained by the grabbing and movement capabilities of the mechanical arm, and the efficiency is not high, and the adaptability and reliability are poor.

[0043] To solve the above problems, the embodiment of the present application provides a sample plate transfer control method, system and related device of an intelligent laboratory, which improves the efficiency and adaptability of the mechanical arm in laboratory automation through scenario-based compliant control and visual-haptic collaborative control, and significantly improves the sample transfer reliability and efficiency in high-throughput experimental scenarios.

[0044] The sample plate transfer control method of the intelligent laboratory provided by the embodiment of the present application can be applied to a humanoid robot system as shown in Figure 1a Please refer to Figure 1a and Figure 1b , Figure 1a is a schematic diagram of an architecture of a humanoid robot system provided by the embodiment of the present application, Figure 1b is another schematic diagram of an architecture of a humanoid robot system provided by the embodiment of the present application; as shown in Figure 1a The humanoid robot system 100 includes a humanoid robot 110, a controller 120, an environment perception and navigation system 130, and a plate station module 140, and the humanoid robot 110 is controlled by the controller 120. As shown in Figure 1b The humanoid robot 110, the controller 120, and the environment perception and navigation system 130 can be integrated.

[0045] In the present scheme, the controller 120 refers to a computer used for processing a large number of computing tasks and storing data. In the present scheme, the controller 120 is deployed with multiple models for executing the sample plate transfer control method of the intelligent laboratory. The controller 120 can also be used to collect data during the use of the model, facilitating subsequent optimization. The environment perception and navigation system 130 provides environment information perception and path planning capabilities for the humanoid robot 110 through the cooperative work of multiple sensors. The environment perception and navigation system 130 includes a vision system, a six-dimensional force sensor, and a navigation positioning system. The vision system is used to obtain visual data, the six-dimensional force sensor is used to obtain mechanical data, and the navigation positioning system is used to provide path information and space information of the intelligent laboratory. The plate station module 140 is a plurality of modular areas for storing sample plates, and the plate station module 140 is a multi-module design. Please refer to Figure 1c , Figure 1cFig. 1 is a structural schematic diagram of a humanoid robot provided by an embodiment of the present application. The humanoid robot 110 comprises a mobile chassis and a mechanical arm. The mechanical arm is provided with a gripping device, which is used to grip materials, and the materials comprise a target sample plate. The mobile chassis comprises a biped structure and / or a wheeled structure. When the humanoid robot 110 moves, the wheeled structure is switched; when the humanoid robot 110 stands, the biped structure is switched. Specifically, when the humanoid robot 110 moves from an initial position to a first target position, from the first target position to a second target position, and from the second target position to other positions, the humanoid robot 110 acts in the wheeled structure; when the humanoid robot 110 pauses at the first target position or the second target position, the humanoid robot 110 stands in the biped structure. The specific size parameters, such as the width and height, of the humanoid robot 110 are associated with the space limitation conditions of the key areas of the intelligent laboratory. For example, if the door width of the intelligent laboratory is 80 cm, the width of the humanoid robot 110 is adjusted to be less than or equal to 75 cm. The humanoid robot 110 can also be provided with an anti-collision device, and the contact force threshold of the anti-collision device can be set according to requirements. For example, the contact force can be greater than or equal to 20 N. The above-mentioned gripping device can be an adaptive electric gripper (supporting 96-well plate / 384-well plate automatic recognition). The clamping force range can be within a preset force range, for example, 0.5-5 N (adjustable). The type of gripping can be replaced according to different application requirements or scenes. Specifically, it can be replaced by magnetic attraction, knob, etc. The above-mentioned mechanical arm can have two, i.e., a dual-arm mechanical arm. The dual-arm mechanical arm has 7 degrees of freedom, including 3 degrees of freedom of the shoulder, 2 degrees of freedom of the elbow, and 2 degrees of freedom of the wrist. The load capacity of the mechanical arm can be 3 kg (continuous) / 5 kg (peak) for a single arm. The working range of the mechanical arm is a dual-arm spread radius of 1.8 m. The humanoid robot 110 is built-in with a 20 Ah lithium battery, and the endurance capacity is 8 hours. The humanoid robot 110 also comprises a head. The mechanical arm is provided with a mechanical arm camera device. The humanoid robot 110 also comprises a head, and the head is provided with a head camera device. Specifically, the vision system mainly comprises at least one head camera device installed on the head of the humanoid robot 110 and at least one mechanical arm camera device at the end of the mechanical arm. The head camera device can be an RGB-D camera (resolution up to 1920x1080). The RGB-D camera of the head can obtain a large range of color images and depth information in real time, which is used to construct a 3D map of the intelligent laboratory, identify the positions of laboratory equipment, and detect obstacles. The mechanical arm camera device at the end of the mechanical arm is used for close-range high-definition shooting and depth perception of the target plate position. In some possible cases, the vision system also comprises a 3D structured light, which can cooperate with the mechanical arm camera device to realize close-range depth perception, and the effective distance can be 0.5-5 m.

[0046] Specifically, the six-axis force sensor is installed at the end of the mechanical arm, and the force resolution reaches 0.1N, which is used to detect the force condition of the humanoid robot 110 end effector when contacting with external objects in three force directions (X, Y, Z axes) and three moment directions (around X, Y, Z axes). During the target sample plate grabbing process, the six-axis force sensor and the vision system data fusion compensate for the vibration error of the humanoid robot 110 body, ensuring accurate grabbing of the grabbing device; during the contact operation process, the force control mode is realized based on the feedback of the force sensor, so that the grabbing device adjusts the clamping force according to the sample plate material, ensuring the operation safety and stability.

[0047] Specifically, the navigation positioning system adopts the simultaneous localization and mapping (SLAM) technology. In the initialization stage of starting the task, the humanoid robot 110 scans the working area through the cooperation of the head 3D camera and the laser radar, constructs a 3D map containing the information of the plate station position, obstacles, laboratory equipment, etc. in real time, determines its position in the map, and judges the obstacle situation in real time. The accuracy of the navigation positioning system is set to ±10mm, which meets the positioning needs of the humanoid robot 110 moving in a large range in the laboratory; combined with path planning algorithms such as A-star algorithm and rapid random exploration tree algorithm, the humanoid robot 110 can realize optimal path planning from the current position to the target position, and dynamically adjust the path according to the environmental changes in the moving process.

[0048] Specifically, the plate station module 140 is a multi-layer plate station built on a C-shaped steel frame, and the height of each layer can be adjusted, specifically between 50-300mm. The plate station module includes an operation area, and at least one positioning code is arranged in each operation area. The positioning code can be a sticker pasted on the operation area, and the distance between each two-dimensional code can be 50cm. The plate station module is configured with a lamp strip for indicating the plate status, such as the empty position. In the plate station module 140, a landmark is arranged on the key path, which can be reflective, luminous or have a specific color, used for navigation with the laser radar.

[0049] Based on this, the application provides a sample plate transfer control method for an intelligent laboratory. The application will be described in detail below with reference to the accompanying drawings.

[0050] Please refer to Figure 2 , Figure 2is a flowchart of a sample plate transfer control method of an intelligent laboratory provided by an embodiment of the present application. The method is applied to a controller of a humanoid robot system, which also includes the humanoid robot. The humanoid robot includes a mechanical arm, which is provided with a gripping device for gripping materials, including a target sample plate. The humanoid robot system also includes a plate station module and an environment perception and navigation system, which includes a first target position and a second target position. As shown in Figure 2 the method includes the following steps:

[0051] S210, obtaining task information and determining initialization task information based on the task information, and controlling the humanoid robot to move from an initial position to the first target position based on the initialization task information.

[0052] The host computer communicates with the humanoid robot system. When the laboratory operator initiates a sample plate transfer task through the host computer, the host computer sends a task instruction to the controller through the Modbus TCP / IP and ROS2 standard interface. The task instruction contains task information, such as the plate station ID of the first target position, which is used to specify the starting storage position of the target sample plate, for example, device A-plate station 1, the plate station ID of the second target position, which is used to specify the placement position of the target sample plate, for example, device B-plate station 3, the type of the target sample plate, for example, 96-well / 384-well, which is used for subsequent adaptive control of the gripping device and selection of operation strategies, priority, for example, high, medium, and low levels, which provides a basis for task scheduling optimization, and high-priority tasks can be executed first.

[0053] After receiving the task instruction, the robot starts the SLAM module immediately in the task initialization stage. At this time, the environment perception and navigation system is started, and environmental images and depth data are collected, which can include information such as plate station positions, laboratory equipment, and obstacles. At the same time, the surrounding environment is scanned in real time at a high frequency to generate a high-precision point cloud map. Through a data fusion algorithm, the visual image data and laser radar point cloud data are integrated to construct a 3D map containing information such as plate station positions, laboratory equipment, and obstacles. After the 3D map is constructed, the controller uses the A-star algorithm for path planning based on the 3D map. The A-star algorithm calculates the optimal path from the current position of the robot to the source plate station by considering the distance cost and heuristic function evaluation value of the path, and reserves a redundant space, which can be 20%, without limitation, to deal with possible sudden obstacles or environmental changes.

[0054] In some possible cases, when the humanoid robot starts from the initial position, the moving chassis of the humanoid robot is switched to the wheeled structure to move along the optimal path at a preset speed. During the movement, the 16-line laser radar continuously scans the surrounding environment at a frequency of 20 Hz to detect obstacle information in real time. Once an obstacle is detected, the laser radar immediately transmits data to the robot control system, triggering the dynamic window algorithm (DWA) to adjust the local path. The DWA algorithm re-plans a safe and feasible local path within a short time (response time < 150 ms) according to the current speed of the robot, the acceleration limit, and the obstacle distribution, ensuring that the robot can flexibly avoid obstacles and continue to move towards the source board station. When the robot reaches a preset stopping distance in front of the first target position, the moving chassis of the humanoid robot is switched to the biped structure to reduce space occupation and adapt to the demand for close-range precise operation. At the same time, the humanoid arm is pre-lifted to a safe height that is a preset safe distance threshold away from the table surface to avoid collision with the table surface or other objects.

[0055] S220, obtaining first sensing data of the humanoid robot at the first target position from the environment perception and navigation system, and determining a grasping error compensation value based on the first sensing data, the first sensing data including first visual data and first mechanical data.

[0056] When the humanoid robot moves to the first target position, the mechanical arm camera at the end of the mechanical arm and the head camera work cooperatively to obtain the first visual data. The mechanical arm camera focuses on the target board to capture high-definition images at a frame rate of not less than 30 Hz, obtaining close-range visual data containing the first positioning code and the edge features of the target sample plate. The head camera synchronously collects the depth information and color images of the surrounding environment to assist in judging the spatial relationship between the robot and the position of the target sample plate. The YOLOv8 model is used to process the images in real time, identify the first positioning code, and output the initial coordinate information of the first positioning code in the robot coordinate system. The six-axis force sensor monitors the interaction force between the gripping device and the outside world in real time. During the approach to the target sample plate, the force data in three force directions (X, Y, and Z axes) and three moment directions (around X, Y, and Z axes) are continuously collected, and the small force fluctuations caused by factors such as robot body vibration and joint motion inertia are recorded. At the same time, the 9-axis IMU synchronously detects the acceleration and angular velocity information of the robot arm to obtain the first mechanical data. Based on the small force fluctuations of the body vibration contained in the first mechanical data, the compensation force value, i.e., the error compensation value, is determined.

[0057] In a possible implementation, the first visual data is from the mechanical arm camera, the acquiring the first sensing data of the humanoid robot at the first target position, and determining a grasping error compensation value based on the first sensing data includes: acquiring first board station coordinate information based on a first positioning code based on the first visual data, the first positioning code being a positioning code of the first target position, the first board station coordinate information being used to indicate accurate coordinates of a board station where the first target position is located; determining a first damping parameter and a first vibration frequency parameter based on the first mechanical data, the first damping parameter and the first vibration frequency parameter being used to represent real-time vibration and damping conditions of a joint of the humanoid robot; determining a first error compensation value based on the first damping parameter and the first board station coordinate information, the first error compensation value including a first target damping coefficient, the first target damping coefficient being used to adjust an error caused by suppressing mechanical arm vibration.

[0058] The first target position is located by real-time detection of the image by the YOLOv8 model, rapid positioning of the first positioning code region, and extraction of the geometric feature points of the positioning code for identification to obtain the accurate coordinates of the first target position under the plate station. The six-dimensional force sensor is used to collect force data in real time at a high sampling rate of 1000 Hz, and the 9-axis IMU is used to monitor the inertial parameters synchronously, and then the first damping parameter and the first error compensation value are obtained by analyzing the vibration and damping characteristics. The specific process can be: performing fast Fourier transform (FFT) on the time domain signal of the six-dimensional force sensor, converting the force data to the frequency domain, and then identifying the peak frequency in the frequency spectrum to determine the main frequency component of the robot joint vibration, i.e., the first vibration frequency parameter. Combined with the robot dynamics model, a mathematical relationship between the force fluctuation amplitude and the joint damping is established. For example, by analyzing the decay rate of the force signal within the vibration period, the equivalent damping coefficient of each joint is calculated, and at the same time, the extended Kalman filter (EKF) is used to filter the force data to separate the damping force component caused by mechanical friction, transmission clearance and other factors, and the first damping parameter is obtained. This parameter reflects the energy loss characteristics of the joint during movement and is used to evaluate the damping state of the current mechanical structure. Then, through data fusion algorithm, the visual positioning information and the mechanical characteristic parameters are combined to generate a vibration compensation strategy, i.e., the first target damping coefficient. Specifically, it can include: based on the first vibration frequency parameter and the first plate station coordinate information, a vibration displacement prediction model of the robot end effector is established. The vibration frequency is mapped to the fluctuation range of the end position using the inverse kinematics of the robot. According to the first damping parameter, the damping capacity of the current joint is evaluated, and combined with the predicted vibration error, the first target damping coefficient required is calculated through the PID control algorithm. For example, if the current damping is insufficient, resulting in excessive vibration error, the target damping coefficient is increased to enhance the damping characteristics of the joint. The first target damping coefficient is converted into the control parameters of the joint servo system to generate the first error compensation value containing the position compensation (ΔX, ΔY, ΔZ) and the attitude compensation (Δθx, Δθy, Δθz).

[0059] As can be seen, in the embodiment, the vibration generated by the mechanical arm during the extension of the mechanical arm is compensated based on the first mechanical data and the first visual data, and the vision and force sense are coordinated to make the operation more stable and accurate.

[0060] S230, based on the first feedback signal, the material grabbing operation of the humanoid robot for the target sample plate is controlled, and the first feedback signal is the second visual data and / or the second mechanical data detected during the material grabbing operation. The material grabbing operation of the humanoid robot is completed through fine tuning based on the first feedback signal.

[0061] The vibration generated in the process of stretching out the mechanical arm of the humanoid robot is compensated in real time based on the calculated grabbing error compensation value, and the first feedback signal is detected in real time, the first feedback signal is used to judge whether the target sample plate will be touched or has been touched, and specifically includes at least one of the second visual data and the second mechanical data, and in this embodiment, the second visual data and the second mechanical data are taken as examples. The second visual data includes data obtained by a mechanical arm camera, and the second mechanical data is data obtained by a six-dimensional force sensor.

[0062] Specifically, the sample plate profile is recognized by an edge detection algorithm and matched with a preset template, and the deviation between the actual grabbing position and the ideal position is calculated. At the same time, the image is analyzed by a deep learning model to determine whether the sample plate is tilted, shifted or abnormal. The six-dimensional force sensor monitors the force data in three force directions (X, Y, Z axes) and three moment directions (around X, Y, Z axes) in real time, and analyzes the time domain change of the force signal to determine whether the grabbing device stably grabs the sample plate. For example, if the force value in a certain direction fluctuates periodically and the amplitude exceeds the threshold value (such as ±0.2N), it indicates that there is vibration or unstable contact, and a mechanical feedback signal is generated. In addition, the 9-axis IMU detects the acceleration and angular velocity information of the mechanical arm to assist in determining the sample plate shaking caused by the movement of the robot body.

[0063] Specifically, if the visual feedback signal shows that the sample plate has an angle deviation, the feedback control module calculates the required joint angle adjustment value according to the deviation, and drives the wrist joint (2 degrees of freedom) to rotate and fine-tune through a PID control algorithm, and the adjustment accuracy can reach ±0.1°. At the same time, combined with the moment data of the six-dimensional force sensor, the grabbing force is kept stable during the adjustment process. When the mechanical feedback signal indicates that the grabbing force is uneven or there is vibration, the system first determines the source of the vibration (such as chassis movement, mechanical arm resonance) through the 9-axis IMU data. If it is the vibration of the mechanical arm itself, the feedback control module adjusts the damping parameters and driving torque of each joint according to the vibration frequency and amplitude to suppress the vibration; if it is a force imbalance caused by position deviation, the position compensation amount is calculated based on the visual data, and the mechanical arm is driven to adjust the small amplitude translation (accuracy ≤0.1mm).

[0064] In a possible embodiment, the second visual data in the first feedback signal is from the mechanical arm camera, the initialization task information further includes a material type, and the material type is used to indicate a characteristic of the target sample plate; the obtaining of the first feedback signal, the fine adjustment based on the first feedback signal, and the completion of the material grabbing operation include: continuously obtaining the first feedback signal during the grabbing process; if it is detected that the first feedback signal meets a first contact condition, adjusting a touch force based on the material type, the first contact condition is used to constrain a relative distance and / or a contact force between the humanoid robot and the target sample plate; determining a first motor torque based on the material type and the touch force, and completing the material grabbing operation based on the first motor torque; determining a current gravity center parameter based on the first feedback signal, the gravity center parameter is used to represent a gravity center of the target sample plate relative to the grabbing of the humanoid robot; determining a second motor torque based on the gravity center parameter, and continuously detecting a gravity center offset, the gravity center offset is used to represent a difference between the gravity center parameter and a target gravity center parameter; and performing gravity center adjustment based on the gravity center offset to smoothly complete the material grabbing operation.

[0065] In the initialization task information, the material, style, and size of the target sample plate can be included, which determines the selection of the grabbing strategy. When controlling the humanoid robot to grab, the first feedback signal (second visual data and second mechanical data) is monitored in real time, and when it is detected that the first feedback signal meets a first contact condition, a corresponding touch force adjustment mechanism is triggered. The first contact condition can be that when the relative distance between the grabbing device and the sample plate is reduced to less than 0.1 cm, and the six-axis force sensor detects that the normal contact force is greater than or equal to 0 N, it is determined that the first contact condition is met. The mapped touch force is determined based on the material type, and different material types are one-to-one corresponding to the touch force, for example, a plastic plate corresponds to 1.5 N, and a glass plate corresponds to 2.5 N. Based on the robot dynamics model and the mechanical structure parameters of the grabbing device, a mathematical relationship between the motor torque and the touch force, and the material characteristics is established. The formula is as follows:

[0066] T = k x F grip x r x μ

[0067] wherein, T is the first motor torque, k is the mechanical transmission coefficient, F gripFor touch intensity, r is the distance from the force point of the grabbing device to the motor shaft, and μ is the material surface friction coefficient, which can be determined by the type of material. The motor torque required to meet the stable grabbing is calculated through the model to ensure that the target sample plate is firmly grabbed. Then, the center of gravity parameters of the sample plate are determined through the IMU and force sensor data, and dynamic balance adjustment is performed; specifically, the 9-axis IMU monitors the acceleration and angular velocity of the robot arm in real time, and the force distribution data detected by the six-dimensional force sensor are combined to calculate the center of gravity position of the sample plate relative to the grabbing device through the mechanical balance equation. The calculated center of gravity parameters are compared with the target center of gravity parameters (the center of gravity position in the ideal balanced state), and the center of gravity offset is calculated. When the offset exceeds the threshold value (such as 2mm), the required angle adjustment amount of each joint is calculated through the inverse kinematics algorithm, and then the second motor torque is determined. The control system drives the joints of the robot arm to move and adjust the arm posture, so that the center of gravity returns to the target range. In this process, the system continuously monitors the center of gravity offset at a frequency of not less than 100Hz, and realizes dynamic closed-loop adjustment through PID control.

[0068] It can be seen that in the present embodiment, the grabbing control process based on the material type and real-time feedback combines mechanical data and visual data, allowing for a small compliant motion during contact to avoid hard collisions, and the robot simultaneously performs plate position recognition and grabbing device pre-adjustment during movement, improving the efficiency and adaptability of the robot arm in laboratory automation and significantly improving the sample transfer reliability and efficiency in high-throughput experimental scenarios.

[0069] S240, detecting the material grabbing state, if it is detected that the material grabbing state is a completed grabbing state, controlling the humanoid robot to move to a second target position based on the initialization task information, and dynamically adjusting the state during the movement based on second feedback information, the second feedback information including third visual data and / or third mechanical data detected during the movement.

[0070] The material grabbing state can be determined by at least one of vision and mechanics. The vision data can be images continuously acquired by the robot head camera and the robot arm camera. The mechanics data can be contact force continuously monitored by the six-axis force sensor between the grabbing device and the target sample plate. The material grabbing state can be determined by one or both of the vision data and the mechanics data. The material grabbing state includes completed grabbing. After confirming the completed grabbing state, the robot moves according to the target plate station ID (i.e., the second target position) in the initialization task information, so that the target sample plate can be smoothly transferred. During the movement, the robot collects third vision data and third mechanics data as second feedback information for dynamic state adjustment. Specifically, the third vision data can be acquired by at least one of the robot head camera and the robot arm camera. The third mechanics data can be acquired by the six-axis force sensor.

[0071] Specifically, the robot head camera acquires environmental images at a frequency of no less than 10 Hz, and combines with the laser radar data to identify the type (such as static equipment and dynamic personnel) and position of the obstacle in real time through a semantic segmentation algorithm. When the obstacle is detected to approach a preset safety distance, the robot adjusts the moving speed and direction according to the DWA algorithm to realize dynamic obstacle avoidance. The robot arm camera continuously monitors the levelness of the sample plate to detect the attitude change at a frequency of no less than 10 Hz. When the third vision data shows that the levelness deviation of the sample plate exceeds 0.5°, the system immediately triggers a micro-adjustment mechanism to restore the attitude of the sample plate to the allowable range by controlling the joint angle of the robot arm, and the adjustment accuracy is ±0.1°. The six-axis force sensor and the 9-axis IMU monitor the force and vibration state of the robot arm in real time. When the third mechanics data shows that there is an abnormal force fluctuation caused by the movement of the chassis or the resonance of the robot arm, the system adjusts the damping parameters and driving torque of each joint according to the vibration frequency and amplitude through the PID algorithm to suppress the vibration. The six-axis force sensor continuously monitors the contact force between the grabbing device and the sample plate. If the contact force deviates from the target touch force by more than the allowable range (such as ±0.2N), the system determines that there is a risk of sample plate loosening, and immediately adjusts the motor torque of the grabbing device to restore the contact force to a stable state, ensuring that the sample plate does not fall off during movement.

[0072] In a possible embodiment, the third visual data is from the head camera and / or the mechanical arm camera, and the state dynamic adjustment based on the second feedback information during movement includes: receiving second feedback information from the environment perception and navigation system; when it is detected based on the second feedback information that a sudden obstacle is encountered, controlling the humanoid robot to brake urgently, and replanning a path based on the environment perception and navigation system; and determining chassis vibration compensation data based on a PID algorithm, determining levelness data based on the second feedback information, the chassis vibration compensation data being used to reduce the degree of vibration of the chassis of the humanoid robot, and the levelness data being levelness data of the target sample plate; and dynamically adjusting the state of the humanoid robot based on the chassis vibration compensation data and the levelness data.

[0073] The head camera collects environmental images at a frequency of no less than 10 Hz, combines laser radar data, and identifies the type (such as static equipment and dynamic personnel) and position of obstacles in real time through a semantic segmentation algorithm; the mechanical arm camera continuously monitors the levelness of the sample plate to detect changes in its posture at a frequency of no less than 10 Hz; and the third visual data is formed based on the above. The six-axis force sensor and the 9-axis IMU monitor the force and vibration state of the mechanical arm in real time to form the third mechanical data. The control system uses a multi-modal fusion algorithm based on deep learning to obtain the second feedback information, aligns the laser radar point cloud with the visual image in space, identifies the obstacle categories (static equipment, dynamic personnel, etc.) using a semantic segmentation technique, and calculates the distance, speed, and other parameters of the obstacles relative to the robot. For sample plate levelness detection, the mechanical arm camera compares real-time pose data with the ideal level state through edge detection and pose estimation algorithms, and outputs levelness data with an accuracy of 0.1°.

[0074] The laser radar and the vision system judge the obstacle risk through a preset threshold. For example, when the distance of the detected front obstacle is less than a safety threshold (such as 0.5 m) and the relative distance exceeds a warning value, an emergency braking instruction is triggered. The control system quickly cuts off the power of the wheeled driving motor, and at the same time, starts the electromagnetic brake device to control the braking distance of the robot within 0.3 m (in the wheeled mode) to avoid collision. After braking, the robot calls the RRT* algorithm for path re-planning based on the real-time map constructed by SLAM. The algorithm takes the current position as the starting point and the target plate station as the ending point, optimizes the path length and smoothness while avoiding obstacles. In this process, the 9-axis IMU monitors the vibration frequency and amplitude of the chassis in real time, and the six-axis force sensor captures the vibration stress transmitted by the robot arm. Based on the PID algorithm, compensation parameters are calculated. For example, when a 15 Hz low-frequency vibration is detected, the PID controller outputs a compensation signal to the joint driver to dynamically adjust the motor damping coefficient and driving torque to suppress vibration transmission. Based on the levelness data obtained by the robot arm camera, the system establishes a closed-loop control loop. When the levelness deviation exceeds a threshold (such as 0.5°), the control system calculates the joint adjustment amount of the robot arm through the inverse kinematics algorithm, and preferentially drives the two degrees of freedom of the wrist to fine-tune the posture, with an adjustment accuracy of ±0.1°. The controller integrates the path re-planning result, vibration compensation parameters, and levelness adjustment instructions, and synchronously issues them to each execution module of the robot through the service interface (such as / motion_control) of the ROS2 framework.

[0075] As can be seen, in the present embodiment, through high-frequency data acquisition and multi-modal fusion algorithm of the laser radar and the vision sensor, the system can detect dynamic obstacles in real time. The chassis vibration compensation mechanism based on the PID algorithm can greatly reduce the end displacement error caused by vibration. The six-axis force sensor monitors the gripping force fluctuation in real time (with an accuracy of ±0.1 N), effectively avoiding sample plate loss. The vision feedback based on the robot arm camera and the inverse kinematics algorithm can quickly correct the levelness deviation and ensure the ideal posture during transfer. In this way, the efficiency and adaptability of the robot arm in laboratory automation are improved, and the sample transfer reliability and efficiency in high-throughput experimental scenarios are significantly improved.

[0076] S250, when it is detected that the humanoid robot reaches the second target position, acquiring third feedback information and placing the target sample plate based on the third feedback information, the third feedback information including fourth vision data and / or fourth mechanical data.

[0077] The second target position is the target transfer position of the target sample plate, and the fourth mechanical data and the fourth vision data in the third feedback information are acquired in the same way as the first feedback information, which will not be described here.

[0078] In a possible implementation, the fourth visual data is from the mechanical arm camera, and the acquiring the third feedback information and placing the target sample plate based on the third feedback information comprises: acquiring second plate station coordinate information based on a second positioning code based on the fourth visual data, the second positioning code being a positioning code of the second target position, the second plate station coordinate information being used to indicate accurate coordinates of a plate station where the first target position is located; acquiring a first empty space condition of a target placement position, the target placement position being an empty space indicated by the initialization task information for placing the target sample plate, the first empty space condition comprising an idle condition; if the first empty space condition is the idle condition, controlling the humanoid robot to perform a material placement operation, in the material placement operation, determining a second damping parameter and a second vibration frequency parameter based on the fourth mechanical data, the second damping parameter and the second vibration frequency parameter being used to represent real-time vibration and damping conditions of a joint of the humanoid robot; determining a second error compensation value based on the second damping parameter and the second plate station coordinate information, the second error compensation value comprising a second target damping coefficient, the second target damping coefficient being used to adjust an error caused by mechanical arm vibration; and continuously acquiring a third feedback signal during placement; if it is detected that the third feedback signal meets a second contact condition, adjusting a touch force based on the material type, the second contact condition being used to constrain a relative distance and / or contact force between the humanoid robot and the target placement position; determining a first motor torque based on the material type and the touch force, and completing the material placement operation based on the first motor torque; when it is detected that the target sample plate has been separated from the grabbing device, controlling the mechanical arm to be lifted and acquiring a target placement position picture of the mechanical arm camera; determining a placement result based on the target position placement picture, the placement result comprising a seated state.

[0079] The third feedback information is obtained in the same way as the first feedback information, and the fourth visual data and the fourth mechanical data are obtained. The fourth visual data includes the second board station coordinate information obtained by scanning the second positioning code, and further includes the first empty space condition. When no obstruction is detected on the target board site, the first empty space condition is determined to be an idle condition, and the information is fed back to the robot control system. Only when the first empty space condition is the idle condition, the robot control system triggers the material placing operation instruction, otherwise the first empty space condition is determined repeatedly or reported. In the material placing operation process, the six-axis force sensor at the end of the mechanical arm and the nine-axis IMU collect the fourth mechanical data in real time, which is used to analyze the joint vibration and damping condition. In the process of material placing operation, the second damping parameter and the second vibration frequency parameter are determined based on the fourth mechanical data, and the second error compensation value is determined based on the second damping parameter and the second board station coordinate information. The process is the same as the process of determining the first damping parameter and the first vibration frequency parameter based on the first mechanical data, and determining the first error compensation value based on the first damping parameter and the first board station coordinate information. Details are not repeated here, and reference can be made to the above detailed description.

[0080] In the placing process, the system collects the third feedback signal (including the fourth visual data and the fourth mechanical data) in real time, which is used for dynamic adjustment of the placing operation: whether it meets the second contact condition is determined, for example, when the relative distance between the grabbing device and the target placing position is reduced to 0.1 cm or less, and the six-axis force sensor detects that the normal contact force is greater than 0N, it is determined that the second contact condition is met. The control system adjusts the corresponding target touch force (such as 1.5N for plastic plate placing and 2.5N for glass plate) from the pre-stored parameter table according to the material type in the initialization task information. Through the PID adjustment mechanism, the contact force feedback by the six-axis force sensor in real time is compared with the target touch force, the driving torque of the grabbing device motor is dynamically adjusted to ensure that the contact force is stable within the allowable fluctuation range (such as ±0.15N), and damage to the target placing position or the sample plate due to excessive torque is avoided. Combined with the physical characteristic parameters (density, elastic modulus, surface friction coefficient, etc.) of the material type, based on the robot dynamics model and the mechanical structure parameters of the grabbing device, a mathematical relationship between the motor torque and the touch force, material characteristics is established, and the second motor torque required for stable placing is calculated. The second motor torque calculated by the model is sent to the grabbing device driving motor to ensure stable placing of the sample plate. When the force sensor detects a sudden change in contact force and the sample plate has been separated from the grabbing device, the mechanical arm is lifted.

[0081] In the mechanical arm lifting process, the mechanical arm camera shoots the target placement position picture, and the placement result is determined through visual image analysis. If the image analysis result shows that the sample plate has been accurately seated at the target position, and the pose error is within the allowed range, it is determined that the placement result is seated, and the robot control system feeds back the result to the upper computer. If the placement fails, the error code (such as E-002: placement offset) is automatically recorded, and the retry strategy (maximum of 3 retries) is generated.

[0082] It can be seen that in the embodiment, through the cooperation of multiple links such as visual positioning, vacancy detection, dynamic parameter adjustment, and contact force control, accurate control of target sample plate placement is realized, the efficiency and adaptability of the mechanical arm in laboratory automation are improved, and the sample transfer reliability and efficiency in the high-throughput experiment scene are significantly improved.

[0083] In a possible embodiment, the target sample plate is provided with a sample label, the sample label is used to display the relevant information of the target sample plate, the vacancy condition further includes a non-idle condition, and the method further includes: obtaining label information of the target sample plate based on the sample label, the label information including relevant information of the target sample plate; determining an information comparison result based on the label information and the initial task information, the comparison result including information consistency and information inconsistency; if the comparison result is the information consistency, controlling the humanoid robot to return to the initial position; if the comparison result is the information inconsistency, reporting an abnormal condition and controlling the humanoid robot to return to the initial position; if the first vacancy condition is the non-idle condition, reporting an abnormal condition and generating a retry strategy, and controlling the humanoid robot to repeatedly obtain a second vacancy condition of the target placement position based on the retry strategy; if the second vacancy condition is the non-idle condition, controlling the humanoid robot to carry the target sample plate to return to the initial position; if the second vacancy condition is the idle condition, controlling the humanoid robot to perform the material placement operation; and when it is determined that the placement result is the seated, controlling the humanoid robot to return to the initial position.

[0084] In a possible embodiment, before the material placement operation is performed, if it is detected that the first vacancy condition is the non-idle condition, the system reports an abnormality (error code E-004), and simultaneously generates a retry strategy. The retry strategy is based on a time window mechanism, for example, retrying the vacancy detection every 30 seconds within 3 minutes, to avoid task failure caused by temporary occupation. Within the retry period, the mechanical arm continuously obtains a second vacancy condition: still non-idle: the mechanical arm carries the sample plate back to the initial position, and the upper computer marks the task as “to be handled by human” and notifies the laboratory manager through message pushing (such as short message, email). Turned to idle: immediately perform the material placement operation to ensure efficient completion of the task.

[0085] Wherein, after detecting that the first empty position is an idle condition and the material placement operation has been completed, the label information of the target sample plate is obtained, wherein each sample plate can be attached with a sample label for scanning, and the scanning result is the relevant information of the sample plate. The label information can be completed by the head camera device or the mechanical arm camera device. The read label information is compared with the initialization task information (such as target plate station ID, material type). If the sample ID in the label matches the target plate station ID, and the material type is consistent with the task requirement, the system determines that the comparison result is “information consistent”, and triggers the mechanical arm to return to the initial position instruction. If the sample ID does not match (such as A plate is placed in B plate position) or the material type conflicts (such as liquid sample is mistakenly placed in solid plate position), the system immediately reports an exception (error code E-003), and prompts the operator through the sound and light alarm device (such as LED lamp group + buzzer), and stores the exception details (such as error type, timestamp, position coordinates) to the log system. The mechanical arm returns to the initial position after completing the exception reporting, avoiding the expansion of errors.

[0086] Wherein, when the system determines through visual analysis that the placement result is “seated”, the mechanical arm camera device collects the target placement position picture again, and verifies the coincidence degree of the sample plate edge and the plate position mark (the requirement deviation is ≤1mm) through the template matching algorithm, and at the same time, combines the force sensor data to confirm that the contact force is stable in the preset range (such as 0.5N±0.1N), double verification to ensure the placement quality. Then control the humanoid robot to return to the initial position, and the system updates the task state to “completed”.

[0087] As can be seen, in the embodiment, by reading the sample label information and comparing it with the initialization task information in real time, the matching of the sample plate and the target position can be accurately identified. When the target placement position is temporarily occupied (such as the experimental personnel temporarily place articles), the system continuously monitors the empty position state through the retry strategy, avoiding the task failure caused by accidental conflict. Through the closed-loop control of “information verification-dynamic fault tolerance-quality verification-automatic regression”, the accuracy, reliability and efficiency of the sample plate transfer task are improved.

[0088] In an antibody screening experiment in a certain biomedical laboratory, the sample plates (containing 96-well plates and 384-well plates) after centrifugation need to be transferred from the high-speed centrifuge to the enzyme label instrument for fluorescence detection. The experimental process includes the following key links: the centrifuge has 8 plate positions, the maximum speed is 15,000 rpm, and the sample plate needs to be detected within 30 minutes after centrifugation. Among them, the enzyme label instrument has 12 plate positions and supports multi-wavelength detection, which needs to be strictly matched with the type of sample plate. The laboratory environment constraints include a laboratory area of 60 square meters, containing 12 devices such as pipettors and incubators, and a passage width of only 1.2 meters. Task initialization stage task receiving and analysis; the experimental personnel configure the task list through the graphical interface, send the host computer instructions to the controller, including: the first target position is the centrifuge (ID: LXJ-01), the second target position is the enzyme label instrument (ID: MTP-02); the target sample plate information is 3 96-well plates (type A) and 2 384-well plates (type B), and the priority is that type A plates are set to high priority because they contain unstable antibody markers; the time constraint is to complete the detection within 30 minutes after centrifugation. After the humanoid robot starts, it scans the laboratory by the head 3D camera for 360°, which takes 45 seconds to generate a point cloud map containing device positions and passage widths. Label key areas: centrifuge hatch position (X=2.5m, Y=1.8m), enzyme label instrument plate area (X=4.2m, Y=3.5m), high-temperature equipment area (X=1.2m, Y=0.8m to X=1.8m, Y=1.5m). Use A-star algorithm to generate a path from the initial position (X=0, Y=0) to the first target position, i.e. the centrifuge, straight along the main passage, then turn right by 90°, then slow down to 0.5m / s through the narrow area, and then arrive at 1m in front of the centrifuge. Pre-plan the mechanical arm motion trajectory, from a safe height (Z=1.2m) to a plate grabbing height (Z=0.3m), and adjust the end-of-arm gripper pose to -45° (adapt to the inclined hatch of the centrifuge). The humanoid robot travels along the planned path at a speed of 1.2m / s, and the laser radar scans the obstacles in front of it in real time. When the pipettor (moving speed 0.6m / s) is detected to enter the safety distance (1.5m), the DWA algorithm is triggered: the predicted collision time is 0.8 seconds, the speed is adjusted to 0.8m / s, and at the same time, it is moved right by 0.2m to maintain a safety distance of 1m. Double-foot standing stabilization: arrive at the centrifuge, switch to double-foot mode, and detect the ground flatness (inclination <3°) through the IMU. The mechanical arm is pre-lifted to Z=1.0m to avoid collision with the hatch. The mechanical arm camera focuses on the target plate (such as the 3rd position of the centrifuge), identifies the first positioning code, i.e. the plate two-dimensional code, through the YOLOv8 model, and obtains the accurate coordinates (X=2.55m, Y=1.82m, Z=0.28m). Combine the post-centrifugation plate position offset compensation algorithm (based on historical data, the average offset is 0.3mm) to correct the target position of the gripper.The grabbing device is lowered at a speed of 0.3 m / s, and a force control mode is triggered when the grabbing device contacts the sample plate. The initial force threshold is 0.5 N (detecting contact), and then the grabbing device is gradually clamped from 1.5 N (for a plastic plate) to 2.0 N (stabilized after 3 seconds). The six-dimensional force sensor monitors the torque change in real time, and the posture of the mechanical arm is adjusted to balance the center of gravity of the sample plate (with a maximum deviation of 5 mm). After the material grabbing operation is completed, the humanoid robot moves to the enzyme marker (i.e., the second target position) at a speed of 0.8 m / s. During the movement, an obstacle is detected, and the incubator door is opened (occupying 1 / 3 of the channel). A bypass path is generated by using the RRT* algorithm, and the time consumed is 1.2 seconds. The speed is reduced to 0.5 m / s during the bypassing to ensure that the deviation of the level of the sample plate is less than 0.8°. The mechanical arm uses zero force control mode, and the joint space impedance controller compensates for the vibration of the chassis at 5-15 Hz. The maximum amplitude compensation is ±2 mm. After reaching the enzyme marker, the camera of the mechanical arm scans the second positioning code (i.e., the plate two-dimensional code) again, and it is found that the cumulative positioning error is 2.8 mm (mainly caused by uneven ground). Visual servo control is triggered, and the mechanical arm is finely adjusted to the target position (X=4.20±0.005 m, Y=3.50±0.005 m). The grabbing device is lowered at a speed of 0.2 m / s, and a pressure of 0.8 N is applied to the enzyme marker plate for 1 second when the grabbing device contacts the enzyme marker plate. After the force sensor detects the pressure mutation, a release instruction is triggered, and the grabbing device is withdrawn at a speed of 0.1 m / s. After the target sample plate is placed, the reader reads the sample label (ID: AB-0012), and compares the sample label with the task list of the upper computer. That is, the initial task information is obtained, including the plate type (96 wells), the experimental project (antibody screening), and the validity period (2024-12-31). Then, the plate state is confirmed. The enzyme marker photoelectric sensor detects that the sample plate is in place (the signal level changes from high to low). Then, the humanoid robot sends a task completion signal (including a time stamp: 14:23:17) to the upper computer.

[0089] In some cases, after the target sample plate is placed, the sample plate label (ID: AB-0012) can also be read by the built-in RFID reader of the enzyme marker at the second target position, and compared with the task list of the upper computer. That is, each plate position can be provided with a tool for reading and scanning the sample label, and each plate position is in communication with the system, so that the information can be shared.

[0090] It should be noted that the task issued by the upper computer to the controller can include multiple subtasks, that is, multiple task information, and the priority needs to be determined. Each task is completed from high to low priority.

[0091] In a possible embodiment, please refer to Figure 3 , Figure 3 is another sample plate transfer control method flowchart of an intelligent laboratory provided by the embodiment of the present application, as Figure 3As shown, first, task initialization and movement are performed. The controller receives the host computer task instruction, parses the initialization task information, including the first target position, the second target position, and other information, constructs an environment map through SLAM and plans a path, and controls the robot to move from the initial position to the first target position. During the movement, obstacle avoidance or path re-planning is performed; then, after reaching the first target position, precision grabbing preparation is performed. The camera at the end of the mechanical arm identifies the first positioning code of the source board station to obtain sub-millimeter level coordinate information; the six-axis force sensor and the IMU detect joint vibration and damping parameters to calculate the grabbing error compensation value (such as the target damping coefficient) and correct the trajectory of the end of the mechanical arm. Then, adaptive grabbing and fine tuning are performed. The gripper descends at a speed of 0.3 m / s, and the force control mode is triggered after contacting the sample board. The clamping force is adjusted according to the material type (such as plastic / glass) (1.5 N / 2.5 N); the IMU monitors the center of gravity shift to dynamically adjust the arm posture to ensure stable grabbing. Then, after successful grabbing, the humanoid robot is controlled to move to the second target position. During the movement, dynamic control and obstacle avoidance or path re-planning are performed. The robot switches to wheel mode and moves. The laser radar detects obstacles in real time and triggers the RRT* algorithm to re-plan the path. The PID algorithm compensates for chassis vibration, and the vision system monitors the sample board level (fine tuning when the deviation is greater than 0.5°). After reaching the second target position, verification and precise placement are performed. The camera identifies the second positioning code, and the photoelectric sensor detects the idle state of the board. If it is idle, precise placement is performed. During the placement process, the damping and torque are dynamically adjusted through mechanical data to avoid impact. The vision confirms that the sample board is completely seated (deviation ≤1 mm). The sample label information is read and compared with the task information. If they are inconsistent, an exception is reported. If the target board is not idle, automatic detection is performed for n times until it is idle or returns to the initial position. If it is idle within n times, precise placement is performed. During the placement process, the damping and torque are dynamically adjusted through mechanical data to avoid impact. The vision confirms that the sample board is completely seated (deviation ≤1 mm). The sample label information is read and compared with the task information. If they are inconsistent, an exception is reported. If it is not idle within n times, an exception is reported and the original position is returned. After the task is completed, the humanoid robot returns to the initial position.

[0092] Please refer to Figure 4 , Figure 4 is a structural schematic diagram of a sample plate transfer control device of an intelligent laboratory according to an embodiment of the present application. The device is applied to a controller of a humanoid robot system, and the humanoid robot system further includes the humanoid robot. The humanoid robot includes a mechanical arm, and the mechanical arm is provided with a grabbing device for grabbing materials, including a target sample plate. The humanoid robot system further includes a board station module and an environment perception and navigation system. The board station module includes a first target position and a second target position; as shown in Figure 4 the sample plate transfer control device 400 of the intelligent laboratory includes:

[0093] The initialization module 410 is configured to acquire task information, determine initialization task information based on the task information, and control the humanoid robot to move from an initial position to the first target position based on the initialization task information.

[0094] The first material grabbing module 420 is configured to acquire first sensing data of the humanoid robot at the first target position from an environment perception and navigation system, and determine a grabbing error compensation value based on the first sensing data, the first sensing data including first visual data and first mechanical data.

[0095] The second material grabbing module 430 is configured to control the humanoid robot to perform a material grabbing operation on a target sample plate based on the grabbing error compensation value, and acquire a first feedback signal, and fine-tune based on the first feedback signal so that the humanoid robot completes the material grabbing operation, the first feedback signal being second visual data and / or second mechanical data detected during the material grabbing operation.

[0096] The material transfer module 440 is configured to detect a material grabbing state, and if the material grabbing state is detected as a completed grabbing state, control the humanoid robot to move to a second target position based on the initialization task information, and dynamically adjust the state based on second feedback information during the movement, the second feedback information including third visual data and / or third mechanical data detected during the movement.

[0097] The material placing module 450 is configured to acquire third feedback information when the humanoid robot is detected to arrive at the second target position, and place the target sample plate based on the third feedback information, the third feedback information including fourth visual data and / or fourth mechanical data.

[0098] In a possible embodiment, the first visual data is from the mechanical arm camera, and the first material grabbing module 420, in the aspect of acquiring first sensing data of the humanoid robot at the first target position and determining a grabbing error compensation value based on the first sensing data, is specifically configured to:

[0099] acquire first plate station coordinate information based on a first positioning code based on the first visual data, the first positioning code being a positioning code of the first target position, the first plate station coordinate information being used to indicate accurate coordinates of a plate station where the first target position is located;

[0100] determine a first damping parameter and a first vibration frequency parameter based on the first mechanical data, the first damping parameter and the first vibration frequency parameter being used to represent real-time vibration and damping conditions of joints of the humanoid robot.

[0101] determine a first error compensation value based on the first damping parameter and the first plate coordinate information, the first error compensation value comprising a first target damping coefficient, the first target damping coefficient being used to adjust an error caused by suppressing mechanical arm vibration.

[0102] In a possible implementation, the second visual data in the first feedback signal is from the mechanical arm camera, and the initialization task information further comprises a material type, the material type being used to indicate a characteristic of the target sample plate; the second material grabbing module 430, in the aspect of acquiring the first feedback signal and fine-tuning the material grabbing operation based on the first feedback signal, is specifically configured to:

[0103] continuously acquire the first feedback signal during the grabbing process;

[0104] if it is detected that the first feedback signal meets a first contact condition, adjust a touch force based on the material type, the first contact condition being used to constrain a relative distance and / or a contact force between the humanoid robot and the target sample plate;

[0105] determine a first motor torque based on the material type and the touch force, and complete the material grabbing operation based on the first motor torque;

[0106] determine a current gravity center parameter based on the first feedback signal, the gravity center parameter being used to represent a gravity center of the target sample plate relative to the grabbing of the humanoid robot;

[0107] determine a second motor torque based on the gravity center parameter, and continuously detect a gravity center offset, the gravity center offset being used to represent a difference between the gravity center parameter and a target gravity center parameter;

[0108] perform gravity center adjustment based on the gravity center offset to smoothly complete the material grabbing operation.

[0109] In a possible implementation, the third visual data is from the head camera and / or the mechanical arm camera, and the material transfer module 440, in the aspect of performing state dynamic adjustment based on second feedback information during the moving process, is specifically configured to:

[0110] receive second feedback information from the environment perception and navigation system;

[0111] when it is detected based on the second feedback information that a sudden obstacle is encountered, controlling the humanoid robot to brake in an emergency and replanning a path based on the environment perception and navigation system; and determining chassis vibration compensation data based on a PID algorithm, determining levelness data based on the second feedback information, the chassis vibration compensation data being used to reduce the degree of vibration of the chassis of the humanoid robot, and the levelness data being levelness data of the target sample plate;

[0112] dynamically adjusting the state of the humanoid robot based on the chassis vibration compensation data and the levelness data.

[0113] In a possible embodiment, the fourth visual data is from the mechanical arm camera, and the material placing module 450 is specifically used for:

[0114] obtaining second plate station coordinate information based on a second positioning code based on the fourth visual data, the second positioning code being a positioning code of the second target position, and the second plate station coordinate information being used to indicate accurate coordinates of a plate station under the first target position;

[0115] obtaining first empty space conditions of a target placement position, the target placement position being an empty space indicated by the initialization task information for placing the target sample plate, and the first empty space conditions including an idle condition;

[0116] if the first empty space conditions are the idle condition, controlling the humanoid robot to perform a material placing operation, in which, based on the fourth mechanical data, a second damping parameter and a second vibration frequency parameter are determined, the second damping parameter and the second vibration frequency parameter being used to represent real-time vibration conditions and damping conditions of the joints of the humanoid robot; based on the second damping parameter and the second plate station coordinate information, a second error compensation value is determined, the second error compensation value including a second target damping coefficient, the second target damping coefficient being used to adjust an error caused by mechanical arm vibration; and a third feedback signal is continuously obtained during the placing process;

[0117] if it is detected that the third feedback signal meets a second contact condition, adjusting a touch force based on the material type, the second contact condition being used to constrain the relative distance and / or contact force between the humanoid robot and the target placement position;

[0118] determining a second motor torque based on the material type and the touch force, and completing the material placing operation based on the second motor torque;

[0119] When it is detected that the target sample plate has been separated from the gripping device, the robotic arm is controlled to be lifted and a target placement position image of the robotic arm camera device is acquired;

[0120] A placement result is determined based on the target location placement picture, where the placement result includes being seated.

[0121] In a possible embodiment, the target sample plate is provided with a sample label, and the sample label is used to display relevant information of the target sample plate. The vacancy situation also includes a non-vacancy situation. The material placement module 450 is further specifically used to:

[0122] acquiring label information of the target sample plate based on the sample label, wherein the label information includes relevant information of the target sample plate;

[0123] determining an information comparison result based on the tag information and the initialization task information, the comparison result including information consistency and information inconsistency; if the comparison result is that the information is consistent, controlling the humanoid robot to return to the initial position; if the comparison result is that the information is inconsistent, reporting an abnormality and controlling the humanoid robot to return to the initial position;

[0124] If the first vacancy situation is the non-idle situation, an abnormal situation is reported and a retry strategy is generated, and based on the retry strategy, the humanoid robot is controlled to repeatedly obtain a second vacancy situation of the target placement position; if the second vacancy situation is the non-idle situation, the humanoid robot is controlled to carry the target sample plate back to the initial position; if the second vacancy situation is the idle situation, the humanoid robot is controlled to perform a material placement operation;

[0125] When it is determined that the placement result is that the humanoid robot has been seated, the humanoid robot is controlled to return to the initial position.

[0126] It is worth noting that the specific functional implementation of the sample plate transfer control device of the intelligent laboratory can be found in the above Figure 2 The description of the sample plate transfer control method of the intelligent laboratory shown in FIG20 is as follows, for example, the initialization module is used to implement the relevant content of executing S210. The various units or modules in the sample plate transfer control device 400 of the intelligent laboratory can be individually or completely merged into one or several other units or modules to form a structure, or one (or some) of the units or modules can be further divided into multiple functionally smaller units or modules to form a structure, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present invention. The above-mentioned units or modules are divided based on logical functions. In actual applications, the functions of one unit (or module) are implemented by multiple units (or modules), or the functions of multiple units (or modules) are implemented by one unit (or module).

[0127] In the case of employing an integrated unit, please refer to Figure 5 , Figure 5 is another structural schematic diagram of a sample plate transfer control device of an intelligent laboratory provided by the embodiment of the present application, as Figure 5 shown, the sample plate transfer control device 400 of the intelligent laboratory comprises a processing module 402 and a communication module 401. The processing module 402 is used for controlling and managing the actions of the sample plate transfer control device 400 of the intelligent laboratory, for example, executing the steps of the initialization module 410, the first material grabbing module 420, the second material grabbing module 430, the material transfer module 440 and the material placing module 450, and / or for executing other processes of the technologies described herein. The communication module 401 is used for the interaction between the sample plate transfer control device 400 of the intelligent laboratory and other devices. As Figure 5 shown, the sample plate transfer control device 400 of the intelligent laboratory can also comprise a storage module 403, which is used for storing the program code and data of the sample plate transfer control device 400 of the intelligent laboratory.

[0128] The processing module 402 can be a processor or a controller, for example, a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It can realize or execute various exemplary logical blocks, modules and circuits described in combination with the disclosure. The processor can also be a combination of computing functions, such as one or more microprocessor combinations, combinations of DSP and microprocessor, etc. The communication module 401 can be a transceiver, RF circuit or communication interface, etc. The storage module 403 can be a memory.

[0129] All related contents of each scene involved in the above method embodiments can be cited to the functional description of the corresponding functional module, which will not be repeated here. The above sample plate transfer control device 400 of the intelligent laboratory can execute the sample plate transfer control method of the intelligent laboratory shown in the above Figure 2 .

[0130] It can be seen that the sample plate transfer control device of the intelligent laboratory described in the embodiments of the present application obtains task information, determines initialization task information based on the task information, and controls the humanoid robot to move from an initial position to a first target position based on the initialization task information. The first sensing data of the humanoid robot at the first target position is obtained from the environment perception and navigation system, and a grasping error compensation value is determined based on the first sensing data. The first sensing data includes first visual data and first mechanical data. The humanoid robot is controlled to perform material grasping operation on the target sample plate based on the grasping error compensation value, and a first feedback signal is obtained. Fine tuning is performed based on the first feedback signal to enable the humanoid robot to complete the material grasping operation. The first feedback signal is second visual data and / or second mechanical data detected during the material grasping operation. The material grasping state is detected. If the material grasping state is detected as a completed grasping state, the humanoid robot is controlled to move to a second target position based on the initialization task information, and state dynamic adjustment is performed based on second feedback information during the movement. The second feedback information includes third visual data and / or third mechanical data detected during the movement. When the humanoid robot reaches the second target position, third feedback information is obtained and the target sample plate is placed based on the third feedback information. The third feedback information includes fourth visual data and / or fourth mechanical data. In this way, through scenario-based compliant control and visual-force coordination control, the efficiency and adaptability of the robotic arm in laboratory automation are improved, and the sample transfer reliability and efficiency in high-throughput experimental scenarios are significantly improved.

[0131] Please refer to Figure 6 , Figure 6 is a structural schematic diagram of an electronic device according to an embodiment of the present application. As shown in the figure, the electronic device 600 includes a processor 610, a memory 620, a communication interface 630, and one or more programs 621. The one or more programs 621 are stored in the memory 620 and are configured to be executed by the processor 610.

[0132] The processor 610, the memory 620, and the communication interface 630 are connected to each other and complete communication work among each other.

[0133] The memory 620 can be a volatile memory such as a dynamic random access memory (DRAM), or a non-volatile memory such as a mechanical hard disk. The memory 620 is used to store a set of executable program codes. The processor 610 is used to call one or more programs 621 stored in the memory 620, and can execute any of the steps of the intelligent laboratory sample plate transfer control method described in the embodiments. Figure 2 The embodiments described above.

[0134] The electronic device 600 can include a controller, etc., which are merely examples and are not exhaustive, including but not limited to the above electronic devices.

[0135] The embodiments of the present application further provide a computer storage medium, which stores a computer program for electronic data exchange, and the computer program causes a computer to execute some or all of the steps of any of the methods described in the above method embodiments. The computer includes an electronic device.

[0136] The embodiments of the present application further provide a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to execute some or all of the steps of any of the methods described in the above method embodiments. The computer program product can be a software installation package, and the computer includes an electronic device.

[0137] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, some steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0138] In the above embodiments, the description of each embodiment is focused on, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0139] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of the above units is only a logical function division. There can be another division manner for actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical or other forms.

[0140] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0141] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0142] When the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable memory. Based on this understanding, the technical solutions of the present application, essentially or the part that contributes to the prior art, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes several instructions for making a computer electronic device (which can be a personal computer, an electronic device or a network electronic device, etc.) execute all or part of the steps of the above-mentioned method of each embodiment of the present application. The aforementioned memory includes: a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0143] A person of ordinary skill in the art can understand that all or part of the steps in the above-mentioned embodiments can be completed by programs instructing relevant hardware, and the programs can be stored in a computer readable memory, which can include: a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0144] The embodiments of the present application are described in detail above, and the principles and implementation manners of the present application are described by applying specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed; in summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A sample plate transfer control method for an intelligent laboratory, characterized in that: A controller for a humanoid robot system, the humanoid robot system further comprising the humanoid robot, the humanoid robot comprising a robotic arm, the robotic arm being provided with a gripping device, the gripping device being used to grip a material, the material comprising a target sample plate, the humanoid robot system further comprising a plate station module and an environmental perception and navigation system, the plate station module comprising a first target position and a second target position; the method comprising: Acquiring task information and determining initialization task information based on the task information, and controlling the humanoid robot to move from an initial position to the first target position based on the initialization task information; Acquiring first sensory data of the humanoid robot at the first target position from the environmental perception and navigation system, and determining a grasping error compensation value based on the first sensory data, the first sensory data including first visual data and first mechanical data; controlling the humanoid robot to perform a material grabbing operation on a target sample plate based on the grabbing error compensation value, and obtaining a first feedback signal; and performing fine-tuning based on the first feedback signal so that the humanoid robot completes the material grabbing operation, wherein the first feedback signal is second visual data and / or second mechanical data detected during the material grabbing operation; detecting a material grasping state, and if it is detected that the material grasping state is a completed grasping state, controlling the humanoid robot to move to a second target position based on the initialization task information, and dynamically adjusting the state based on second feedback information during the movement, wherein the second feedback information includes third visual data and / or third mechanical data detected during the movement; When it is detected that the humanoid robot reaches the second target position, third feedback information is obtained and the target sample plate is placed based on the third feedback information, where the third feedback information includes fourth visual data and / or fourth mechanical data.

2. The method according to claim 1, characterized in that The first visual data comes from the robotic arm camera device, and the acquiring first sensor data of the humanoid robot at the first target position and determining the grasping error compensation value based on the first sensor data includes: Acquire first board station coordinate information based on a first positioning code based on the first visual data, where the first positioning code is a positioning code of the first target position, and the first board station coordinate information is used to indicate the precise coordinates of the board station where the first target position is located; Determining a first damping parameter and a first vibration frequency parameter based on the first mechanical data, wherein the first damping parameter and the first vibration frequency parameter are used to characterize a real-time vibration condition and a damping condition of a joint of the humanoid robot; A first error compensation value is determined based on the first damping parameter and the first panel station coordinate information. The first error compensation value includes a first target damping coefficient. The first target damping coefficient is used to adjust and suppress errors caused by vibration of the robotic arm.

3. The method according to claim 1 or 2, characterized in that The second visual data in the first feedback signal comes from the robotic arm camera device, and the initialization task information also includes a material type, where the material type is used to indicate characteristics of the target sample plate. The obtaining of the first feedback signal and performing fine-tuning based on the first feedback signal to complete the material grabbing operation include: Continuously obtaining a first feedback signal during the grasping process; If it is detected that the first feedback signal meets a first contact condition, adjusting the touch force based on the material type, the first contact condition being used to constrain the relative distance and / or contact force between the humanoid robot and the target sample plate; Determining a first motor torque based on the material type and the touch force, and completing the material clamping operation based on the first motor torque; determining a current center of gravity parameter based on the first feedback signal, the center of gravity parameter being used to characterize the center of gravity of the target sample plate relative to the gripper of the humanoid robot; determining a second motor torque based on the center of gravity parameter and continuously detecting a center of gravity offset, wherein the center of gravity offset is used to represent a difference between the center of gravity parameter and a target center of gravity parameter; The center of gravity is adjusted based on the center of gravity offset to smoothly complete the material grabbing operation.

4. The method according to claim 1, wherein The robotic arm is provided with a robotic arm camera device, the humanoid robot further includes a head, the head is provided with a head camera device, the third visual data comes from the head camera device and / or the robotic arm camera device, and the dynamic state adjustment based on the second feedback information during the movement includes: receiving second feedback information from the environmental perception and navigation system; When encountering a sudden obstacle is detected based on the second feedback information, the humanoid robot is controlled to perform emergency braking and replan a path based on the environmental perception and navigation system; and chassis vibration compensation data is determined based on a PID algorithm, and levelness data is determined based on the second feedback information, wherein the chassis vibration compensation data is used to reduce the degree of chassis vibration of the humanoid robot, and the levelness data is the levelness data of the target sample plate; The state of the humanoid robot is dynamically adjusted based on the chassis vibration compensation data and the levelness data.

5. The method according to claim 1, wherein The fourth visual data comes from the robotic arm camera device, and the obtaining of third feedback information and placing the target sample plate based on the third feedback information includes: Acquire, based on the fourth visual data, second board station coordinate information based on a second positioning code, where the second positioning code is a positioning code of the second target position, and the second board station coordinate information is used to indicate precise coordinates of the board station where the first target position is located; Acquire a first vacancy status of a target placement position, where the target placement position is a vacancy indicated by the initialization task information for placing the target sample plate, and the first vacancy status includes an idle status; If the first vacancy condition is the idle condition, controlling the humanoid robot to perform a material placement operation, during the material placement operation, determining a second damping parameter and a second vibration frequency parameter based on the fourth mechanical data, the second damping parameter and the second vibration frequency parameter being used to characterize the real-time vibration and damping conditions of the joints of the humanoid robot; determining a second error compensation value based on the second damping parameter and the second plate station coordinate information, the second error compensation value including a second target damping coefficient, the second target damping coefficient being used to adjust and suppress errors caused by vibration of the robot arm; and continuously obtaining a third feedback signal during the placement process; If it is detected that the third feedback signal meets a second contact condition, adjusting the touch force based on the material type, the second contact condition being used to constrain the relative distance and / or contact force between the humanoid robot and the target placement position; determining a second motor torque based on the material type and the touch force, and completing the material placement operation based on the second motor torque; When it is detected that the target sample plate has been separated from the gripping device, the robotic arm is controlled to be lifted and a target placement position image of the robotic arm camera device is acquired; A placement result is determined based on the target location placement picture, where the placement result includes being seated.

6. The method according to claim 5, characterized in that The target sample plate is provided with a sample label, the sample label being used to display relevant information of the target sample plate, the vacancy situation also including a non-vacant situation, and the method further comprising: acquiring label information of the target sample plate based on the sample label, wherein the label information includes relevant information of the target sample plate; determining an information comparison result based on the tag information and the initialization task information, the comparison result including information consistency and information inconsistency; if the comparison result is that the information is consistent, controlling the humanoid robot to return to the initial position; if the comparison result is that the information is inconsistent, reporting an abnormality and controlling the humanoid robot to return to the initial position; If the first vacancy situation is the non-idle situation, an abnormal situation is reported and a retry strategy is generated, and based on the retry strategy, the humanoid robot is controlled to repeatedly obtain a second vacancy situation of the target placement position; if the second vacancy situation is the non-idle situation, the humanoid robot is controlled to carry the target sample plate back to the initial position; if the second vacancy situation is the idle situation, the humanoid robot is controlled to perform a material placement operation; When it is determined that the placement result is that the humanoid robot has been seated, the humanoid robot is controlled to return to the initial position.

7. A humanoid robot system using the intelligent laboratory sample plate transfer control method according to any one of claims 1 to 6, characterized in that: include: A humanoid robot comprising a mobile chassis and a robotic arm, wherein the robotic arm is provided with a gripping device, wherein the gripping device is used to grip materials, wherein the materials include the target sample plate; the mobile chassis comprises a bipedal structure and / or a wheeled structure, wherein the wheeled structure is switched to when the humanoid robot is moving, and the bipedal structure is switched to when the humanoid robot is standing; A controller, configured to execute the method according to any one of claims 1 to 6; An environmental perception and navigation system, comprising a visual system, a six-dimensional force sensor, and a navigation and positioning system. The visual system is used to acquire visual data, the six-dimensional force sensor is used to acquire mechanical data, and the navigation and positioning system is used to provide path information and spatial information of the intelligent laboratory. The board station module is a plurality of modular areas for storing sample plates, and the board station module is a multi-modular design.

8. The system according to claim 7, characterized in that The visual system includes a plurality of camera devices, and the plurality of camera devices include a head camera device and a robotic arm camera device; The six-dimensional force sensor includes a mechanical arm mechanical sensor for sensing the force applied to the gripping device and / or the end of the mechanical arm; the board station module is a multi-layer board station constructed with a C-shaped channel steel frame, and the board station module includes an operating area, each of which is provided with at least one positioning code; The controller, the head camera device, the robotic arm camera device, and the six-dimensional force sensor are synchronized in time and space; The robotic arm of the humanoid robot includes a robotic arm with 7 degrees of freedom.

9. A sample plate transfer control device for an intelligent laboratory, characterized in that: A controller for a humanoid robot system, the humanoid robot system further comprising the humanoid robot, the humanoid robot comprising a robotic arm, the robotic arm being provided with a gripping device, the gripping device being used to grip a material, the material comprising a target sample plate, the humanoid robot system further comprising a plate station module and an environmental perception and navigation system, the plate station module comprising a first target position and a second target position; the device comprising: an initialization module, configured to obtain task information and determine initialization task information based on the task information, and control the humanoid robot to move from an initial position to the first target position based on the initialization task information; a first material grasping module, configured to obtain first sensor data of the humanoid robot at the first target position from an environmental perception and navigation system, and determine a grasping error compensation value based on the first sensor data, wherein the first sensor data includes first visual data and first mechanical data; a second material grasping module, configured to control the humanoid robot to perform a material grasping operation on a target sample plate based on the grasping error compensation value, obtain a first feedback signal, and perform fine-tuning based on the first feedback signal so that the humanoid robot completes the material grasping operation, wherein the first feedback signal is second visual data and / or second mechanical data detected during the material grasping operation; a material transfer module, configured to detect a material grasping state, and if the material grasping state is detected to be a completed grasping state, control the humanoid robot to move to a second target position based on the initialization task information, and dynamically adjust the state based on second feedback information during the movement, wherein the second feedback information includes third visual data and / or third mechanical data detected during the movement; A material placement module is used to obtain third feedback information and place the target sample plate based on the third feedback information when detecting that the humanoid robot reaches the second target position, wherein the third feedback information includes fourth visual data and / or fourth mechanical data.

10. An electronic device, characterized in that: The method comprises a processor and a memory storing execution instructions, wherein the memory stores one or more programs; when the processor executes the execution instructions stored in the memory, the processor executes the method according to any one of claims 1 to 6.

Images