Control method for sample moving device, and sample moving device and system

Abstract

A control method for a sample moving device, and a sample moving device, which can achieve calibration of a robotic arm with low cost and with a simple structure. The control method comprises: an acquisition step: by means of a camera, acquiring an image containing both a current position of a first marker and a current position of a second marker; a determination step: on the basis of position related information between the first marker and the second marker in the image, and relative position information between the first marker and the second marker when the first marker is at a target position, determining a difference between the current position of the first marker and the target position, and, on the basis of the difference, determining the amount of movement to be performed by the first marker; a first movement step: on the basis of said amount of movement, controlling a robotic arm so as to move the first marker from the current position to an updated position; and a determination step: determining whether a difference between the updated position and the target position is less than or equal to a preset threshold, and if the difference is greater than the threshold, executing an acquisition step.

Description

Control method for sample moving device, sample moving device and system Technical Field

[0001] This application relates to the field of scientific instruments, and more specifically to a control method, a sample moving device, and a sample moving system for a sample moving device. Background Technology

[0002] It is known that a sample moving device, including a robotic arm, is used to transport a sample-bearing object. During movement, the robotic arm may experience positional deviations due to collisions, vibrations, manufacturing errors, etc., preventing it from reaching the desired position (the sample's location) to operate on the object. Therefore, calibration is necessary to ensure that the actuators mounted on the robotic arm accurately move to the position of the object and perform the corresponding operations. Typically, as preparation for calibration, a single visual marker (e.g., QR code, ARCIO code, etc.) is used to calibrate the robotic arm or actuator to determine its position and orientation. Existing calibration methods include, for example, hand-eye calibration. Specifically, hand-eye calibration determines the position and orientation of the camera relative to the actuator mounted on the robotic arm, i.e., calculating the transformation matrix between the camera coordinate system and the robotic arm coordinate system. This enables the transformation from the camera coordinate system to the robotic arm coordinate system, guiding the robotic arm to complete various operational tasks. For example, when an object to be moved enters the camera's field of view, the image including the object is analyzed to determine its specific position and orientation in the camera coordinate system. Then, using the transformation matrix obtained earlier through hand-eye calibration, the corresponding position of the object in the robotic arm's coordinate system is obtained. Next, based on the hand-eye calibration results, the desired position the robotic arm needs to reach is calculated, and based on the calculation results, the robotic arm is moved to the desired position, thereby enabling the actuators to manipulate the object located at the corresponding position.

[0003] However, because the kinematic equations of many robotic arms are unknown or extremely complex, it is impossible to control the movement of the robotic arm based on the kinematic equations alone. Therefore, it is necessary to guide the actuators mounted on the robotic arm to the location of the object to be moved through methods such as teaching and reproduction, force and tactile feedback, path planning and optimization. Furthermore, complex path planning can also make it difficult for the robotic arm to move to the desired position in one go along the planned path.

[0004] Furthermore, existing calibration methods may require the camera's position and orientation to remain constant, or the camera's position and orientation relative to the robotic arm to remain constant. However, even after the camera has been calibrated, factors such as vibration and collisions may cause deviations in the camera's position and orientation, making it impossible to accurately manipulate the moving object. Summary of the Invention

[0005] This application was developed to address the aforementioned technical problems, and its purpose is to provide a control method for a sample moving device that enables robotic arm calibration at a lower cost while reducing dependence on camera position. Based on this, this application further provides a sample moving device and a sample moving system.

[0006] One technical solution of this application provides a control method for a sample moving device, the sample moving device including a camera, a robotic arm and an actuator connected to the robotic arm, the actuator being used to enable the moving object used to place the sample to move.

[0007] The control method includes an acquisition step, a determination step, a first movement step, and a judgment step.

[0008] In the acquisition step, an image containing the current positions of both the first marker and the second marker is acquired by a camera, wherein the first marker is arranged to be fixed relative to the actuator, and the position of the second marker is arranged to be fixed relative to the target position.

[0009] In the determination step, based on the positional information between the first marker and the second marker in the image, and the pre-stored relative positional information between the first marker and the second marker when the first marker is located at the target position, the difference between the current position of the first marker and the target position is determined.

[0010] In addition, in the determination step, the amount of the first marker to be moved is determined based on the difference.

[0011] In the first movement step, the robotic arm is controlled based on the amount to be moved to move the first marker from its current position to the updated position.

[0012] In the judgment step, it is determined whether the difference between the updated position and the target position is less than or equal to a preset threshold. If it is greater than the threshold, the acquisition step is executed.

[0013] According to the control method for a sample moving device described in this technical solution, by setting two different markers—a first marker fixed relative to the actuator and a second marker fixed relative to the target position—the calibration of the robotic arm can be decoupled from the camera position. Specifically, regardless of how the camera position changes, the positional information between the first and second markers remains constant at any given moment. Furthermore, the pre-stored relative positional information between the first and second markers when the first marker is at the target position is also a fixed value independent of coordinate system changes. Therefore, regardless of whether the camera position changes, the difference between the current position of the first marker and the target position can be determined based on the aforementioned positional and relative positional information. In addition, by iteratively bringing the first marker closer to the target position, complex control logic and drive units are not required, enabling efficient and cost-effective robotic arm calibration.

[0014] Optionally, the location-related information is the relative position information between the first marker and the second marker in the image.

[0015] According to the control method for the sample moving device described in this technical solution, the difference between the current position of the first marker and the target position can be determined based on the relative position information between the first marker and the second marker in the image and the relative positional relationship between the two when the first marker is located at the target position.

[0016] Optionally, the difference includes difference components across multiple degrees of freedom.

[0017] Optionally, the determination step further includes: determining the degree of freedom that satisfies the preset conditions among multiple degrees of freedom as the determined degree of freedom, and determining the amount to be moved of the first marker based on the difference component of the difference in the determined degree of freedom.

[0018] According to the control method for the sample moving device described in this technical solution, the amount to be moved of the first marker is determined by the difference component of the difference in multiple degrees of freedom that meet preset conditions. This enables the first marker to move only in the degrees of freedom that meet the preset conditions, reducing the complexity of the control logic.

[0019] Optionally, the determination step further includes: determining whether the condition that the difference between the updated position and the target position on each of the plurality of degrees of freedom is less than or equal to the component of the threshold on that degree of freedom is met; if the condition is not met, then the acquisition step is performed.

[0020] According to the control method for the sample moving device described in this technical solution, by decomposing the complex motion into simple motions in each degree of freedom and judging the motion in each degree of freedom, the judgment logic can be simplified.

[0021] Optionally, the degree of freedom is a translational degree of freedom. Furthermore, the determining step further includes: determining the one with the largest difference among the plurality of translational degrees of freedom as the determined degree of freedom.

[0022] According to the control method for the sample moving device described in this technical solution, by controlling the robotic arm based solely on translational degrees of freedom, calibration can be performed with simpler control logic, and drive units (e.g., motors) with only translational degree of freedom driving functions can be used, thus reducing costs. Furthermore, since each control operation targets only one translational degree of freedom, the control logic can be simplified to the greatest extent, allowing for the use of multiple drive units with only one translational degree of freedom driving functions, thereby further reducing costs.

[0023] Optionally, the sample moving device also includes a motor for driving the robotic arm to move. The degrees of freedom are translational degrees of freedom. The control method includes a second moving step before the first execution of the acquisition step. In the second moving step, the robotic arm is controlled to move the first marker from the origin position to an initial position, where the origin position is a fixed position and the initial position is the current position of the first marker before the first execution of the first moving step. After the decision step, the control method also includes a recording step, which includes recording the change in motor parameters corresponding to the first marker from the origin position to the updated position.

[0024] Optionally, after the recording step, a third movement step is also included, in which the actuator is moved by controlling the robotic arm based on the change in motor parameters.

[0025] According to the control method for the sample moving device described in this technical solution, by using the change in motor parameters corresponding to the movement of the first marker from the initial position to the updated position, the calibration can be completed by directly using the stored change in motor parameters to drive the robotic arm to move in translational degrees of freedom without repeating the acquisition step, determination step, first moving step, and judgment step during the next calibration.

[0026] Optionally, multiple second markers are fixed relative to the target position, with the multiple second markers located on opposite sides of the target position.

[0027] According to the control method for a sample moving device described in this technical solution, by fixing multiple second markers relative to a target position in a manner that clamps the target position, calibration can be performed based on the multiple second markers, thereby improving calibration accuracy.

[0028] Optionally, the control methods include, in sequence:

[0029] The first stage includes the acquisition step, the determination step, the first movement step, and the judgment step. The target position is a first target position. Before the acquisition step is executed for the first time, the first stage also includes a second movement step. In the second movement step, the robotic arm is controlled to move the first marker from the origin position to the initial position. The origin position is a fixed position, and the initial position is the current position of the first marker before the first movement step is executed for the first time in the first stage. After the judgment step in the first stage, a first information recording step is also included. In the first information recording step, first control information corresponding to the displacement of the first marker from the origin position to the updated position is recorded.

[0030] Move the first marker back to its original position from its updated position; and

[0031] The second stage includes the second movement step, the acquisition step, the determination step, the first movement step, and the judgment step. The target position is a second target position that is different from the first target position. After the judgment step, the second stage also includes a second information recording step. In the second information recording step, second control information corresponding to the displacement of the first marker from the origin position to the updated position is recorded.

[0032] According to the control method for the sample moving device described in this technical solution, calibration can be performed on different target positions using the same method. Furthermore, the recorded first and second control information can be used for subsequent control.

[0033] Optionally, after the second stage, the control method further includes controlling the robotic arm based on the first control information and the second control information, so that the first marker moves from the updated position in the first information recording step to the updated position in the second information recording step, or moves from the updated position in the second information recording step to the updated position in the first information recording step.

[0034] According to the control method for the sample moving device described in this technical solution, the first marker can be directly positioned in two different locations using the first control information and the second control information obtained through calibration, which can be directly applied to sample handling tasks.

[0035] Optionally, the first target location and its corresponding second marker in the first stage are located in the first layer of the multi-layer sample holder. The second target location and its corresponding second marker in the second stage are located in the second layer of the multi-layer sample holder. After the second stage, the control method further includes:

[0036] - The second position is determined based on the first updated position, the second updated position, and pre-stored positional information between the third layer and the first and second layers in a multi-layered system. The first updated position is the updated position in the first information recording step, the second updated position is the updated position in the second information recording step, and the second position is the position in the third layer where the first marker is to be reached; and

[0037] - Third movement step: In the third movement step, the robotic arm is controlled based on the first control information, the second control information, the second position, and the pre-stored relative position information between the first position and the initial position, so that the first marker moves from the first position to the second position. The first position is the position of the first marker before the third movement step is executed.

[0038] According to the control method for the sample moving device described in this technical solution, the sample shovel can accurately pick up the object to be moved located on each layer based directly on the first control information and the second control information obtained through calibration, as well as the position-related information between the layers of the sample holder.

[0039] On the other hand, this application also provides a sample moving device, including a camera, an actuator, and a robotic arm. The camera is used to capture an image that simultaneously includes the current position of a first marker and the current position of a second marker. The actuator is used to enable movement of an object to be moved for placing a sample. The robotic arm is connected to the actuator. The first marker is positioned at a fixed position relative to the actuator, and the second marker is positioned at a fixed position relative to a target position. Furthermore, the sample moving device includes an acquisition module, a first determination module, a second determination module, a control module, and a judgment module. The acquisition module is configured to acquire an image. The first determination module is configured to determine the difference between the current position of the first marker and the target position based on positional information between the first and second markers in the image acquired by the acquisition module, and pre-stored relative positional information between the first and second markers when the first marker is at the target position. The second determination module is configured to determine the amount to be moved of the first marker based on the difference determined by the first determination module. The control module is configured to control the robotic arm based on the amount to be moved determined by the second determination module to move the first marker from its current position to the updated position. The judgment module is configured to determine whether the difference between the updated position and the target position is less than or equal to a preset threshold. If it is greater than the threshold, the acquisition module, the first determination module, the second determination module, and the control module are executed repeatedly until it is determined to be less than or equal to the threshold. Attached Figure Description

[0040] Figure 1 is a schematic diagram of a sample moving system that applies a control method for a sample moving device according to an embodiment of this application.

[0041] Figure 2 is a schematic diagram of the sample moving device shown in Figure 1.

[0042] Figure 3 is a schematic diagram showing the software configuration of the sample moving device.

[0043] Figure 4 is a flowchart illustrating a control method for a sample moving device according to the first embodiment of this application.

[0044] Figure 5 is a schematic diagram showing the relative positional relationship between the first marker and the second marker when the first marker is located at the target position.

[0045] Figure 6 is a schematic diagram showing an example of a second marker arranged on a calibration disk.

[0046] Figure 7 is an image showing both the first and second markers located at the current position.

[0047] Figure 8 is a flowchart illustrating a first variation of the control method of the first embodiment.

[0048] Figure 9 is a flowchart illustrating a second variation of the control method of the first embodiment.

[0049] Figure 10 is a flowchart illustrating a third variation of the control method of the first embodiment.

[0050] Figure 11 is a flowchart illustrating a control method for a sample moving device according to a second embodiment of this application.

[0051] Figure 12 is an illustrative diagram showing calibration for two different target locations.

[0052] Figure 13 is a schematic diagram illustrating the sample handling process.

[0053] Figure 14 is an explanatory diagram illustrating the execution of the sample handling task.

[0054] Figure 15 is an illustrative diagram showing the removal of a sample from one layer of the sample holder. Detailed Implementation

[0055] It should be noted that the accompanying drawings are not all drawn to scale, but are enlarged to illustrate various aspects of the invention, and the drawings should not be construed as restrictive.

[0056] Terminology Definition

[0057] In this application, "sample moving device" refers to a scientific instrument used to move a movable object on which a sample can be placed. It can be a sample loading device or other scientific instruments besides a sample loading device.

[0058] In this application, "object to be moved" refers to an object that can be used to place a sample. For example, it can be a sample container for storing samples, a tray for placing sample containers, or other types of objects to be moved, as long as they can be used to place samples.

[0059] In this application, an "actuator" is a component connected to a robotic arm for picking up and moving an object, and it can be of various types. In the following embodiments and variations of this application, a sample shovel is used as an example of an actuator, but the application is not limited to this. The actuator can also be other types such as a gripper (claw) or a suction cup.

[0060] In this application, "target position" refers to the position where the first marker is expected to reach, for example, a fixed position pre-set before calibration. When the first marker is located at the target position, the actuator used to pick up and move the object can accurately operate the object.

[0061] In this application, "origin position" refers to the reference position of the first marker determined by a sensor installed on the sample moving device, which is a fixed position relative to the sample moving position. That is, once the sample moving device is installed, the origin position is a fixed position.

[0062] In this application, "current position" refers to the position of the first marker when the camera takes a picture.

[0063] In this application, "initial position" refers to the current position of the first marker when the acquisition step is first executed in the control method of the following embodiments and their variations.

[0064] "Degree of freedom" is a concept in the field of mechanics, referring to the number of independent motion parameters given to a mechanism when it has a definite motion. In this application, "degree of freedom" refers to a robotic arm having two or more independent motion parameters given to it when it has a definite motion. That is to say, the "degree of freedom" mentioned in this application refers to the number of parameters or directions in which the robotic arm can move independently.

[0065] In this application, "first marker" refers to a visual marker used to determine the current position (or actual position) of the robotic arm (or actuator), such as an ArUco code, but it can also be other types of visual markers. For this purpose, the first marker is arranged to be fixed relative to the actuator.

[0066] In this application, the "second marker" is a visual marker used to mark the target location, such as an ArUco code, but it can also be other types of visual markers. Therefore, the second marker is arranged to be fixed relative to the target location.

[0067] Sample moving system S

[0068] Figure 1 shows a schematic diagram of a sample moving system S using a control method for a sample moving device according to embodiments and variations of this application. As shown in Figure 1, the sample moving system S includes a sample moving device O, a calibration disk 6, and a sample holder 7 for placing the calibration disk 6. Furthermore, as shown in Figure 1, the sample holder 7 has multiple layers, with the calibration disk 6 placed on the bottom layer and the third layer counting from the top, and second markers A2 arranged on the calibration disk 6. Figure 6 shows an example of the arrangement of the second markers A2 on the calibration disk 6. In Figure 6, four second markers A2 are arranged symmetrically about the axis of symmetry of the long side of the rectangular calibration disk 6. However, it should be noted that the arrangement of the second markers A2 is not limited to this, and other arrangements are also possible. In addition, a first marker A1 is arranged on the sample shovel 2, which serves as an actuator in one example of the sample moving device O.

[0069] In this application, the example of a sample shovel 2 serving as an actuator is used to illustrate the arrangement of the first marker A1, but the application is not limited to this. For example, a support could be provided on the sample shovel 2, and the first marker A1 could be arranged on the support. In other words, it is sufficient to arrange the first marker A1 at a fixed position relative to the sample shovel 2 serving as an actuator.

[0070] Furthermore, this application describes the case where the calibration disk 6 is placed in two layers within the sample holder 7, but it is not limited to this; it may also be placed in only one layer within the sample holder 7. Alternatively, the calibration disk 6 may not be placed within the sample holder 7, as long as it is positioned at a fixed target position relative to the first marker A1. Since the second marker A2 is positioned on the calibration disk 6, it can also be said that the second marker A2 is positioned at a fixed target position relative to the first marker A1. Furthermore, this application describes the case where the second marker A2 is positioned on the calibration disk 6, but it is not limited to this. It is acceptable as long as the second marker A2 is positioned at a fixed target position relative to the first marker A1. Therefore, the sample moving system S may not include the sample holder 7.

[0071] Furthermore, this application uses the example of having multiple (specifically four) second markers A2 fixedly arranged relative to a target location, but the number of second markers A2 is not limited to this. For example, only one second marker A2 may be arranged relative to a target location.

[0072] Hardware configuration of sample moving device O

[0073] Figure 2 shows the main components of the sample moving device O from two different perspectives. As shown in Figure 2, the sample moving device O includes a camera 1, a sample shovel 2 (an example of an actuator), a robotic arm 3, a rotating tower 4 (an example of a robotic arm support), and a base 5. Here, the sample shovel 2 is only one example of an actuator; as explained in the "Terminology Definitions" section, the actuator can also be other types such as a suction cup or gripper. The base 5 forms the base portion of the sample moving device O and is used to mount the rotating tower 4. The rotating tower 4 is mounted on the base 5 in a manner that allows it to rotate relative to the base 5, and the rotating tower 5 is used to connect to and support the robotic arm 3. As shown in Figure 1, in this embodiment, the robotic arm 3 is a multi-joint robotic arm, including a first joint 31, a shoulder 32, a second joint 33, a first arm 34, a third joint 35, a second arm 36, and a fourth joint 37. The first joint 31 connects the rotating tower 4 and the shoulder 32 and is connected to a motor (not shown). Driven by a motor, the first joint 31 causes the shoulder 32 and even the entire robotic arm 3 to move relative to the rotating tower 4 in one or more degrees of freedom. In this embodiment, movement in multiple degrees of freedom is described as an example. The second joint 33 is a rotary joint connected to one end of the shoulder 32 and is capable of rotating relative to the shoulder 32. One end of the first arm 34 is connected to the second joint 33, and the other end is connected to the third joint 35. The third joint 35 is a rotary joint and is capable of rotating relative to the first arm 34. One end of the second arm 36 is connected to the second joint 35 and is capable of rotating relative to the first arm 34 via the second joint 36. The fourth joint 37 is a rotary joint connected to the other end of the second arm 36 and is capable of rotating relative to the second arm 36. As an example of an actuator, the sample shovel 2 is mounted on the fourth joint 37 and can rotate relative to the second arm 36 via this joint 37, and can rotate relative to the first arm 34 via the second joint 33, and can rotate relative to the second arm 36 via the third joint 35. In addition, as shown in FIG1, the camera 1 is mounted on the shoulder 32. In other words, camera 1 is fixed relative to shoulder 32 and even the entire robotic arm 3. However, the camera 1 is not limited to this; it can also be set in a movable manner relative to robotic arm 3, or it can be set independently of robotic arm 3.

[0074] Here, we have used a multi-joint robotic arm 3 as an example for illustration, but the type of robotic arm 3 is not limited to this. For example, robotic arm 3 can also be a Cartesian coordinate robotic arm, a cylindrical coordinate robotic arm, a polar coordinate robotic arm, a parallel robotic arm, a SCARA robotic arm, or a collaborative robotic arm.

[0075] Software configuration of sample moving device O

[0076] Figure 3 shows a schematic diagram of the software configuration of the sample moving device O shown in Figure 2. As shown in Figure 3, the sample moving device O includes an acquisition module M1, a first determination module M2, a second determination module M3, a control module M4, and a judgment module M5 as software components.

[0077] The acquisition module M1 is configured to acquire an image captured by camera 1 that simultaneously contains the current positions of the first marker A1 and the second marker A2. It is important to emphasize that "simultaneously contains" means that the first marker A1 and the second marker A2 must both exist in the same image. In other words, the image captured by camera 1 must simultaneously contain both the first marker A1 and the second marker A2.

[0078] The first determining module M2 is configured to determine the difference between the current position of the first marker A1 and the target position based on the position-related information between the first marker A1 and the second marker A2 in the image acquired by the acquisition module M1, and the pre-stored relative position information between the first marker A1 and the second marker A2 when the first marker A1 is located at the target position.

[0079] The second determining module M3 is configured to determine the amount to be moved of the first marker A1 based on the difference determined by the first determining module M2.

[0080] The control module M4 is configured to control the robotic arm 3 based on the amount of movement to be determined by the second determining module M3, so that the first marker A1 is moved from its current position to the updated position.

[0081] The judgment module M5 is configured to determine whether the difference between the updated position and the target position is less than or equal to a preset threshold. If it is greater than the threshold, it requests the repeated execution of the acquisition module M1, the first determination module M2, the second determination module M3, and the control module M4 until it is determined that the difference is less than or equal to the threshold. The threshold will be explained in conjunction with the judgment steps in the following implementation method and its variations.

[0082] The specific functions of each of the above software modules and the meanings of some of the terms will be explained in detail in conjunction with the steps of the control method of an embodiment of this application.

[0083] First Implementation of the Control Method

[0084] Figure 4 shows a flowchart of the control method for the sample moving device O according to the first embodiment of this application. It should be noted that the control method is not only applicable to the sample moving device O described above, but also to sample moving devices of other types or structures.

[0085] Obtain step ST1

[0086] First, in step ST1, the camera 1 acquires an image that simultaneously contains the current position of the first marker A1 and the current position of the second marker.

[0087] Specifically, before camera 1 takes its first picture, a second movement step is performed, controlling robotic arm 3 to move from the origin position to the initial position of the first marker A1. As explained in the "Terminology Definitions" section, the "origin position" referred to here is the reference position of the first marker A1, determined by sensors installed on the sample moving device O, and is a fixed position relative to the sample moving device. Next, camera 1 takes a picture to acquire an image that simultaneously includes the first marker A1 and the second marker A2 at their current positions.

[0088] Figure 7 shows an image containing both a first marker A1 and a second marker A2, illustrating their current positions. For the initial capture, the current position of the first marker A1 is the initial position. As explained in the "Terminology Definitions" section, "initial position" refers to the current position of the first marker when the acquisition step is first executed in the control method described in this embodiment, its variations, and other embodiments. Furthermore, in this embodiment, as shown in Figure 7, there are four second markers A2, with the first marker A1 in the middle and two second markers A2 on either side of it.

[0089] It should be noted that in this embodiment, the example given is that the second movement step is performed before the camera 1 takes its first picture, so that the first marker A1 moves from the origin position to the initial position, but this is not the only possibility. If the camera can capture an image that simultaneously includes the first marker A1 and the second marker A2 when the first marker A1 is at the origin position, then the second movement step may not be performed. In this case, the origin position is the initial position.

[0090] It should be noted that the number of images is not limited to one; multiple images can be captured consecutively, and subsequent steps can be performed based on these multiple images.

[0091] Determine step ST2

[0092] Next, in step ST2, the difference between the current position of the first marker A1 and the target position is determined based on the positional information between the first marker A1 and the second marker A2 in the image, and the pre-stored relative positional information between the first marker A1 and the second marker A2 when the first marker A1 is located at the target position.

[0093] Then, in the determination step ST2, the amount to be moved of the first marker is determined based on the difference.

[0094] Pre-stored relative position information

[0095] The following explains the "relative position information between the first marker A1 and the second marker A2 when the first marker A1 is located at the target position".

[0096] Figure 5 shows the relative position information of the first marker A1 and the second marker A2 when the first marker A1 is located at the target position. In fact, a three-dimensional model of the sample moving device O and even the sample moving system S has been pre-designed before performing the determination step ST2. In the three-dimensional model, the first marker A1 is located at the target position, and the calibration disk 6, on which the second marker A2 is arranged, is set at a fixed position relative to this target position. The position of the first marker A1 in Figure 5 is the target position of the first marker A1 in the three-dimensional model. This target position is the position where the first marker A1 is expected to reach. When the first marker A1 is located at the target position, the sample shovel 2 can accurately operate the object to be moved.

[0097] Since the three-dimensional model of the sample moving device S is pre-designed, as one method, the positional information of the first marker A1 and the second marker A2 when the first marker A1 is at the target position is pre-stored. Therefore, based on the positional information of the first marker A1 and the second marker A2, the relative positional information between the first marker A1 and the second marker A2 can be directly determined. Alternatively, the relative positional information between the first marker A1 and the second marker A2 when the first marker A1 is at the target position is pre-stored.

[0098] As a storage format, the transformation matrix from the local coordinate system of the first marker A1 to the local coordinate system of the second marker A2 is pre-stored when the first marker A1 is located at the target position. Details about the transformation matrix will be explained in the next section.

[0099] It should be noted that since both the first marker A1 and the second marker A2 are at least two-dimensional visual markers, the relative position information between them typically includes the relative translation (or shift) information between the center point of the first marker A1 and the center point of the second marker A2, as well as the relative rotation information between the first marker A1 and the second marker A2. However, the reference for the relative translation information may not be the center point but a corner point. For ease of explanation, the center point is used as an example in this application. Furthermore, the relative rotation information between the first marker A1 and the second marker A2 typically refers to the relative rotation information between the local coordinate system of the first marker A1 and the local coordinate system of the second marker A2. In addition, if the center point of the first marker A1 and the center point of the second marker A2 are respectively set as the origin of their respective local coordinate systems, then the relative translation information between the center point of the first marker A1 and the center point of the second marker A2 refers to the relative translation information between the local coordinate system of the first marker A1 and the local coordinate system of the second marker A2.

[0100] Additionally, preferably, as shown in FIG5, when there are multiple second markers A2, these second markers A2 are arranged symmetrically about the central axis passing through the first marker A1.

[0101] Transformation matrix between two markers

[0102] Based on a pre-designed three-dimensional model of the sample moving device O, the transformation relationship between the first marker A1 and each second marker A2 can be predetermined and stored. This transformation relationship is represented by the following transformation matrix T:

[0103] in:

[0104] The submatrix R located in the upper left corner describes the relative rotation relationship between the other markers;

[0105] The submatrix t in the upper right corner describes the relative translational relationship between the two markers. More specifically, t1, t2, and t3 describe the relative positional relationship between the two markers in the three translational degrees of freedom (or the three translational directions).

[0106] In the case of multiple second markers A2, to distinguish each second marker A2, their ID numbers are labeled as 0, 1, 2, and 3 respectively. Thus, the transformation matrix between the first marker A1 and each second marker A2 in Figure 5 is represented by the following matrix:

[0107] Furthermore, from a coordinate system perspective, the aforementioned transformation matrix represents the transformation from the local coordinate system of the first marker A1 to the local coordinate system of each of the second markers A2. By performing this transformation, the first marker A1 and the second marker A2 can be placed in the same coordinate system, namely, the local coordinate system of each of the second markers A2. In other words, using the aforementioned transformation matrix, the position and orientation of the first marker A1 in the local coordinate system of each of the second markers A2 when the first marker A1 is located at the target position can be determined.

[0108] Location-related information

[0109] The "positional information related to the first marker A1 and the second marker A2 in the image" mentioned above includes the following:

[0110] (1) The relative position information between the first marker A1 and the second marker A2 in the image pixel coordinate system of the image;

[0111] (2) The pixel coordinate information of the first marker A1 and the second marker A2 in the image pixel coordinate system of the image.

[0112] Transformation operation from image pixel coordinate system to camera coordinate system

[0113] For the acquired image, visual marker detection is first performed to identify the first marker A1 and the second marker A2 in the image. Since the visual marker detection uses a well-known detection algorithm, its description is omitted here.

[0114] Then, based on the positional information between the first marker A1 and the second marker A2 in the captured image, and using the solvePnP algorithm, the transformation matrix T, which transforms the local coordinate system of the first marker A1 and the local coordinate systems of each of the second markers A2 to the camera coordinate system, can be calculated. ci (i = 0, 1, 2, 3, 4). Where i = 0, 1, 2, 3 are the ID numbers of the four second markers A2, and i = 4 is the ID number of the first marker A1. That is to say, T ci The four transformation matrices (i = 0, 1, 2, 3) represent the positions and orientations of the four second markers A2 in the camera coordinate system, T c4 This transformation matrix represents the position and orientation of the first marker A1 located at the current position in the camera coordinate system.

[0115] After calculating the above transformation matrix T ci Subsequently, since the target position is a fixed position and the second marker A2 is arranged to be fixed relative to the target position, the transformation matrix T is then used to... ci and transformation matrix T i4rMultiplying (i = 0, 1, 2, 3) allows us to calculate the transformation matrix T from the local coordinate system to the camera coordinate system for each of the second markers A2. c4ri The transformation matrix represents the position and orientation of the first marker A1 in the camera coordinate system with respect to each of the second markers A2. c4ri (i = 0, 1, 2, 3) is specifically represented by the following formula: T c4r0 =T c0 *T 04r T c4r1 =T c1 *T 14r T c4r2 =T c2 *T 24r T c4r3 =T c3 *T 34r

[0116] In this embodiment, since there are four second markers A2 involved in the calibration, the calculated positions and orientations of the first markers A1 in the camera coordinate system are also four. Therefore, these four calculation results can be averaged or weighted averaged to determine a new transformation matrix.

[0117] The transformation matrix It represents the position and orientation of the first marker A1 located at the target position in the camera coordinate system.

[0118] Determine the difference between the current position and the target position of the first marker A1.

[0119] After obtaining the transformation matrix T c4 and Next, first apply the transformation matrix T c4 Inverse is obtained by inverting it. Then, use the inverse matrix Left multiplication matrix The new transformation matrix T is obtained. 44r :

[0120] Transformation matrix T 44r It represents the position and orientation of the first marker A1 at the current position relative to the first marker A1 at the target position. In other words, the transformation matrix T 44r It represents the difference between the current position of the first marker A1 and the target position.

[0121] Transformation matrix T 44r Including the rotation submatrix R 44r Translational submatrix t 44rThrough well-known mathematical operations, the rotation submatrix R can be... 44r The components in the equation are converted into three rotational degrees of freedom components r. x ,r y ,r z These rotational degrees of freedom components can be represented using Euler angles or axis angles. On the other hand, the translational submatrix t 44r The three components represent the three translational degrees of freedom components t. x ,t y ,t z .

[0122] Therefore, it can be understood that the difference between the current position and the target position of the first marker A1 includes difference components in multiple degrees of freedom. Of course, it is obvious that the dimensions of the difference components in the rotational degrees of freedom are different from those in the translational degrees of freedom, and they cannot be directly compared.

[0123] Determine the amount to be moved for the first marker A1.

[0124] As mentioned above, the amount to be moved by the first marker A1 is determined based on the difference between its current position and the target position. Here, "amount to be moved" refers to the amount by which the first marker A1 needs to be moved from its current position to the target position. However, also as mentioned above, since the rotational and translational degrees of freedom have different dimensions, the amount to be moved must be divided into the amount to be moved in the rotational degrees of freedom and the amount to be moved in the translational degrees of freedom.

[0125] Specifically, in this embodiment, the difference components in the three rotational degrees of freedom are synthesized, and the synthesized quantity is used as the amount to be moved in the rotational degrees of freedom. Furthermore, the difference components in the three translational degrees of freedom are synthesized, and the synthesized quantity is used as the amount to be moved in the translational degrees of freedom.

[0126] First moving step ST3

[0127] In the first movement step ST3, the robotic arm 3 is controlled to move the first marker A1 from its current position to the updated position based on the amount to be moved determined in the determination step ST2.

[0128] Specifically, based on the determination of the amount to be moved in the rotational degree of freedom and the amount to be moved in the translational degree of freedom of the first marker A1 in step ST2, the robotic arm 3 can be controlled first based on the amount to be moved in the rotational degree of freedom to make the first marker A1 move from its current position. Then, the robotic arm 3 can be controlled again based on the amount to be moved in the translational degree of freedom to make the first marker A1 move further to reach the updated position.

[0129] Judgment step ST4

[0130] In step ST4, it is determined whether the difference between the updated position reached by the first marker A1 and the target position is less than or equal to a preset threshold. If it is less than or equal to the threshold, control is terminated or other steps are executed. If it is greater than the threshold, step ST1 (acquisition), step ST2 (determination), and step ST3 (first movement) are repeated until it is determined to be less than or equal to the threshold.

[0131] Specifically, after moving the first marker A1 to the updated position according to the amount to be moved, camera 1 can take another picture to obtain a new image that simultaneously includes the first marker A1 and the second marker A2 at the updated position. The difference between the updated position reached by the first marker A1 and the target position is determined in the same manner as in step ST2 described above. Here, "difference" refers to the combined amount of the difference components in the three rotational degrees of freedom and the combined amount of the difference components in the three translational degrees of freedom. Correspondingly, the "preset threshold" refers to the rotational threshold corresponding to the combined amount of the difference components in the three rotational degrees of freedom and the translational threshold corresponding to the combined amount of the difference components in the three translational degrees of freedom.

[0132] If the resultant quantity of the difference components in the three rotational degrees of freedom is less than or equal to the preset rotational threshold and the resultant quantity of the difference components in the three translational degrees of freedom is less than or equal to the preset translational threshold, then control is terminated or other steps are executed.

[0133] On the other hand, if it is determined that the sum of the difference components in the three rotational degrees of freedom is greater than a preset rotational threshold or the sum of the difference components in the three translational degrees of freedom is greater than a preset translational threshold, then the acquisition step ST1 is repeated. However, it should be noted that the updated position of the first marker A1 becomes the "current position" when the acquisition step ST1 is repeated. Then, the determination step ST2, the first movement step ST3, and the judgment step ST4 are executed again until it is determined in a certain round of judgment steps that the difference between the updated position and the target position is less than or equal to a preset threshold.

[0134] Technical effects of this embodiment

[0135] According to the control method described in this embodiment, firstly, by simultaneously employing two visual markers with different arrangements, the calibration of the robotic arm is decoupled from the position and attitude of the camera. This allows the robotic arm to be calibrated without being affected by changes in the camera's position and attitude. Secondly, by controlling the movement of the robotic arm iteratively to gradually bring the first marker closer to the target position, compared to the prior art which attempts to make the first marker reach the target position in one go along a planned path, the control logic and the complexity of the control equipment are significantly simplified.

[0136] First variation of the implementation of the control method

[0137] Figure 8 shows a flowchart of a first variation of the control method of the above embodiment. Here, to avoid repetition, only the differences from the above embodiment will be described.

[0138] The first variation differs from the above-described embodiment in that it includes a determination step ST2A, a first movement step ST3A, and a judgment step ST4A instead of the determination step ST2, the first movement step ST3, and the judgment step ST4.

[0139] Determine step ST2A

[0140] In step ST2A, only the difference components in the three rotational degrees of freedom are synthesized, and the synthesized quantity is used as the amount to be moved by the first marker A1. Alternatively, only the difference components in the three translational degrees of freedom are synthesized, and the synthesized quantity is used as the amount to be moved by the first marker A1. That is, in this first variation, only the synthesized quantity of the difference components related to the rotational degrees of freedom or the synthesized quantity of the difference components related to the translational degrees of freedom is used as the amount to be moved by the first marker A1.

[0141] First moving step ST3A

[0142] In the first movement step ST3A, the robotic arm 3 is controlled to move the first marker A1 from its current position to the updated position based on the amount to be moved of the first marker A1 determined in the determination step ST2A. The amount to be moved of the first marker A1 is the composite amount of the difference components related to the rotational degrees of freedom or the composite amount of the difference components related to the translational degrees of freedom.

[0143] Judgment step ST4A

[0144] In step ST4A, it is determined whether the difference between the updated position and the target position is less than or equal to a preset threshold. Here, the "preset threshold" refers to the rotation threshold corresponding to the resultant amount of the difference components on the three rotational degrees of freedom, or the translation threshold corresponding to the resultant amount of the difference components on the three translational degrees of freedom.

[0145] Technical Effects of the First Modification

[0146] According to the control method described in this variation, since only rotational or translational degrees of freedom are considered, the control logic can be further simplified.

[0147] Second variation of the implementation of the control method

[0148] Figure 9 shows a flowchart of a second variation of the control method of the above embodiment. Here, to avoid repetition, only the differences from the above embodiment will be described.

[0149] The second variation differs from the above embodiment in that it includes a determination step ST2B, a first movement step ST3B, and a judgment step ST4B instead of the determination step ST2, the first movement step ST3, and the judgment step ST4.

[0150] Determine step ST2B

[0151] In step ST2B, the degrees of freedom that satisfy preset conditions among multiple degrees of freedom are first determined as the determined degrees of freedom. Then, the amount of movement to be made by the first marker A1 is determined based on the difference components of the difference in the determined degrees of freedom. Specifically, in this second variation, instead of considering the overall difference in the rotational degrees of freedom and / or the overall difference in the translational degrees of freedom, the difference components in each rotational degree of freedom and / or the difference components in each translational degree of freedom are considered.

[0152] Assuming we only consider the difference component in each translational degree of freedom, the same applies to rotational degrees of freedom. The "preset condition" mentioned here can refer to whether the difference component is the largest. That is, in step ST2B, from multiple (either three or two) translational degrees of freedom, the translational degree of freedom that satisfies the preset condition of "maximum difference component" is determined as the determined degree of freedom. Then, the amount of movement to be made by the first marker A1 is determined based on the difference component of the difference in the determined degree of freedom. For example, assuming the difference component in the first translational degree of freedom is the largest among the three translational degrees of freedom, the first translational degree of freedom is taken as the determined degree of freedom, and the amount of movement to be made by the first marker A1 is determined based on the difference component in the first translational degree of freedom. In this case, the amount of movement to be made by the first marker A1 is actually the amount of movement to be made in the first translational degree of freedom.

[0153] It should be noted that the preset conditions are not limited to the types mentioned above and can also be other types of conditions. For example, the preset condition could also refer to whether the difference component is greater than a preset upper limit value. Assuming we only consider the difference component in each translational degree of freedom, the same applies to rotational degrees of freedom. In step ST2B, the translational degrees of freedom that satisfy the preset condition "the difference component is greater than the upper limit value" are determined from multiple (either three or two) translational degrees of freedom. Then, the amount of movement to be made by the first marker A1 is determined based on the difference component of the difference in the determined degrees of freedom. For example, assuming the difference component between the first and second translational degrees of freedom is greater than the upper limit value, the first and second translational degrees of freedom are taken as the determined degrees of freedom, and the amount of movement to be made by the first marker A1 is determined based on the difference component between the first and second translational degrees of freedom. At this time, the amount to be moved by the first marker A1 includes two parts: the first part is the amount to be moved by the first marker A1 in the first translational degree of freedom, and the second part is the amount to be moved by the first marker A1 in the second translational degree of freedom.

[0154] First moving step ST3B

[0155] In the first movement step ST3B, the robotic arm 3 is controlled based on the amount of movement to be made of the first marker A1 in the determined degrees of freedom so that the first marker A1 moves from the current position to the updated position.

[0156] Determine step ST4B

[0157] In step ST4B, it is determined whether the condition is met that the difference between the updated position and the target position in the degrees of freedom determined above is less than or equal to a preset threshold component in that degree of freedom. If this condition is met, control ends or other steps are executed further. On the other hand, if this condition is not met, step ST1 (acquisition), step ST2B (determination), and step ST3B (first movement) are repeated until the condition is met. The "preset threshold" mentioned here refers to the rotational threshold corresponding to the sum of the difference components in the three rotational degrees of freedom, and / or the translational threshold corresponding to the sum of the difference components in the three translational degrees of freedom.

[0158] Similarly, assuming only translational degrees of freedom are considered, the same applies to rotational degrees of freedom. In this case, the judgment step ST4B determines whether the difference between the updated position and the target position in the translational degrees of freedom is less than or equal to the preset threshold component in that translational degree of freedom.

[0159] However, the threshold can also be preset for each degree of freedom. In this case, the determination step ST4B checks whether the difference between the updated position and the target position in the determined degree of freedom is less than or equal to the preset threshold in the determined degree of freedom.

[0160] Technical Effects of the Second Modification

[0161] According to the control method described in this variation, unlike the above-described embodiments and the first variation, the difference components of the entire rotational degrees of freedom or the entire translational degrees of freedom are not considered. Instead, the degrees of freedom that satisfy preset conditions are used as the determined degrees of freedom. Furthermore, after finding the determined degrees of freedom, the amount of movement to be made by the first marker A1 in that determined degree of freedom is determined based on the difference components of the difference components in that determined degree of freedom, and the first marker A1 is further moved to the updated position based on this amount of movement. Thus, the amount of movement to be made and the direction of movement to be made by the first marker A1 can be specifically determined according to preset requirements, further simplifying the control logic and thereby further reducing the cost of control. Furthermore, since it is not necessary to perform composite calculations for each degree of freedom, the control cycle is shortened, saving computational costs.

[0162] Third variation of the implementation of the control method

[0163] Figure 10 shows a flowchart of a third variation of the control method of the above embodiment. The third variation is derived from the second variation, and differs from the second variation in that it includes a determination step ST4C.

[0164] In step ST4C, it is determined whether the condition is met that the difference between the updated position and the target position in each of the multiple degrees of freedom is less than or equal to the threshold component in that degree of freedom. If this condition is met, control ends or other steps are executed further. On the other hand, if this condition is not met, the acquisition step ST1, the determination step ST2B, and the first movement step ST3B are repeated until the condition is met.

[0165] Technical Effects of the Second Modification

[0166] According to the control method described in this variation, by judging whether the difference between the updated position of the first marker and the target position in each of the multiple degrees of freedom is less than or equal to a preset threshold, and if it is judged that the difference between the two in any degree of freedom is greater than the threshold, the first marker A1 is moved further to reduce the difference between the two, which can ensure that the final position of the first marker A1 is closer to the target position and further improve the calibration accuracy.

[0167] Second implementation of the control method

[0168] Figure 11 shows a flowchart of the control method for the sample moving device O according to the second embodiment of this application. Here, to avoid repetition, only the differences from the first embodiment and the first to third modifications described above will be explained.

[0169] The difference between this second embodiment and the first to third modifications described above is that it further includes a recording step ST5 following the determination step. The recording step ST5 is performed when the determination step in the first embodiment and each modification determines that the threshold value is less than or equal to the threshold value.

[0170] Record step ST5

[0171] In recording step ST5, the change in motor parameters of the motor corresponding to the movement of the first marker A1 from the origin position to the final updated position is recorded. As explained in the "Terminology Definitions" section, the "origin position" refers to the reference position of the first marker determined by the sensor installed on the sample movement device, which is a fixed position relative to the sample movement position. The "final updated position" refers to the updated position determined in the judgment step to be less than or equal to a preset threshold from the target position; it is the position reached by the first marker A1 after the last execution of the first movement step. The "motor parameters" can be the number of microsteps of the motor, the rotation angle information of the motor shaft, or the encoder value corresponding to the rotation angle information.

[0172] Specifically, when considering only multiple translational degrees of freedom, the recorded changes in motor parameters include the parameter changes corresponding to the displacements of the first marker A1 from the origin position in each translational degree of freedom. For example, when there are three translational degrees of freedom, the recorded changes in motor parameters include the parameter changes corresponding to the displacements of the first marker A1 from the origin position in the first translational degree of freedom, the parameter changes corresponding to the displacements of the first marker A1 from the origin position in the second translational degree of freedom, and the parameter changes corresponding to the displacements of the first marker A1 from the origin position in the third translational degree of freedom.

[0173] Technical Effects of the Second Embodiment

[0174] According to the control method described in this embodiment, by recording the change in motor parameters of the motor corresponding to the updated position of the first marker A1, which moves from the origin to a position where the difference between the target position and the target position is less than or equal to a preset threshold, the robotic arm can be directly controlled based on the recorded change in motor parameters to move the first marker A1 to a position where the difference between the target position and the target position is less than or equal to the preset threshold during the next control cycle. This eliminates the need to repeat the acquisition step, determination step, first movement step, and judgment step. Therefore, the preparation process before each handling operation can be significantly simplified.

[0175] In particular, considering only multiple translational degrees of freedom, the robotic arm can be moved along each translational degree of freedom simply by measuring the recorded changes in motor parameters. This significantly simplifies the control logic and allows the use of inexpensive motors, further reducing costs.

[0176] It should be noted that as long as the origin and target positions remain unchanged, automatic calibration can be performed directly based on the recorded changes in motor parameters. This is because the aforementioned changes in motor parameters are based on the aforementioned origin and target positions.

[0177] A variation of the second embodiment of the control method

[0178] The difference from the second embodiment described above is that the control method further includes a third movement step after the recording step. In the third movement step, the robotic arm is controlled to move the sample shovel 2 based on the changes in motor parameters recorded in the recording step ST5.

[0179] Specifically, when the first marker A1 moves back to its original position and then needs to move again to a position where the difference from the target position is less than or equal to a preset threshold, the robotic arm is directly controlled to move the sample shovel 2 based on the change in motor parameters recorded in recording step ST5. Thus, the first marker A1 can be moved to the updated position without executing the acquisition step, determination step, first movement step, and judgment step.

[0180] Calibration for two target locations

[0181] Hereinafter, referring to FIG12, the calibration of two target positions based on the control method of the above embodiment and its modifications will be described. FIG12 shows the first target position 100, the second target position 200, and the origin position 00.

[0182] The calibration for the two target positions includes, in sequence, a first stage of calibration for the first target position 100, a return step, and a second stage of calibration for the second target position 200.

[0183] Phase 1

[0184] In the first stage, the acquisition step, determination step, first movement step, and judgment step of the control method described in any of the first embodiment, the first to third variations, and the second embodiment are executed to move the first marker A1 from the origin position 00 to an updated position where the difference between the marker A1 and the first target position 100 is less than or equal to a preset threshold, and this updated position is recorded as the first updated position 300. Specifically, the first stage also includes the second movement step described above before the first execution of the acquisition step. In the second movement step, the robotic arm is controlled to move the first marker A1 from the origin position 00 to the initial position. However, if the camera 1 can capture an image containing both the first marker A1 and the second marker A2 when the first marker A1 is at the origin position 00, the second movement step may not be required.

[0185] In addition, the first stage includes a first information recording step after the judgment step. In the first information recording step, first control information corresponding to the displacement of the first marker A1 from the origin position 00 to the first updated position 300 is recorded.

[0186] Return to step

[0187] After completing the first information recording step, the robotic arm is controlled to move the first marker A1 from the first updated position 300 back to the original position 00.

[0188] Phase Two

[0189] In the second stage, the acquisition step, determination step, first movement step, and judgment step of the control method described in any of the first embodiment, the first to third variations, and the second embodiment are executed to move the first marker A1 from the origin position 00 to an updated position where the difference between it and the second target position 200 is less than or equal to a preset threshold, and this updated position is recorded as the second updated position 400. Specifically, the second stage also includes the second movement step described above before the first acquisition step is executed. However, if the camera 1 can capture an image containing both the first marker A1 and the second marker A2 when the first marker A1 is at the origin position 00, the second movement step may not be required.

[0190] Furthermore, the second stage includes a second information recording step after the judgment step. In the second information recording step, first control information corresponding to the displacement of the first marker A1 from the origin position 00 to the second updated position 400 is recorded.

[0191] Application scenario—sample handling

[0192] Next, based on the completion of the calibration task for the two target locations shown in Figure 12, the process of sample handling will be described with reference to Figures 13 and 14.

[0193] Figure 13 shows a schematic diagram of a sample handling task, and Figure 14 shows a schematic diagram of the execution of a sample handling task. As shown in Figure 13, the sample handling task refers to using the sample shovel 2 of the sample moving device O to pick up the object to be moved (W) containing the sample from the sample holder 7 (e.g., a sample tray) and place the object to be moved into the injector 8, which is an example of the destination for the object to be moved, or to pick up the object to be moved from the injector 8 and place it into the sample holder 7. Figure 14 shows the origin position 00, the first target position 100, the second target position 200, the first updated position 300, the second updated position 400, and the first position 500. The first target position 100 is located at the sample holder 7 used to place the object to be moved, and the second target position 200 is located at the injector 8, which is an example of the destination for the object to be moved. The first position 500 is the starting position of the first marker A1 before the robotic arm 3 begins performing the sample handling task. The relative position information between the first position 500 and the origin position 00 can be predetermined or pre-stored through the robotic arm control information corresponding to the movement of the first marker A1 from the origin position 00 to the first position 500. As for the first updated position 300 and the second updated position 400, they have been determined through the calibration task for the two target positions shown in Figure 12.

[0194] First, based on the first control information recorded in the first information recording step and the relative position information between the first position 500 and the origin position 00, the robotic arm 3 is controlled to move, so that the first marker A1 moves from the first position 500 to the first updated position 300. After reaching the first updated position 300, the robotic arm 3 and the sample shovel 2 are further controlled to move the object to be moved, which can carry the sample. Next, based on the first control information recorded in the first information recording step and the second control information recorded in the second information recording step, the robotic arm 3 is controlled to move, so that the first marker A1 moves from the first updated position 300 to the second updated position 400, so that the object to be moved carrying the sample and the sample shovel 2 move together to the predetermined position of the sample injector 8. Then, the sample handling task is completed after the sample is removed from the object to be moved W. At this time, the sample shovel 2 holds the object to be moved after the sample has been removed.

[0195] After completing the sample handling task, the robotic arm 3 is again controlled to move based on the first and second control information so that the first marker A1 moves from the second updated position 400 to the first updated position 300. Optionally, after reaching the first updated position 300, the robotic arm 3 and the sample shovel 2 are further controlled to return the object to be moved W to the sample holder 7.

[0196] Application scenario—Retrieving samples from multi-layer sample racks

[0197] Next, based on the calibration task for the two target locations shown in Figure 12, the process of taking samples from any layer of the multi-layered sample holder 7 will be described with reference to Figure 15.

[0198] Assume that each layer of the sample rack 7 shown in Figure 15 contains an object to be moved, on which a sample is placed. Figure 15 shows the origin position 00, the first target position 100, the second target position 200, the first updated position 300, the second updated position 400, the first position 500, and the second position 600. As shown in Figure 15, the first target position 100 and its corresponding second marker A2 are located on the first layer L1 of the multi-layer sample rack 7, and the second target position 200 and its corresponding second marker A2 are located on the second layer L2 of the multi-layer sample rack 7. Except for the first layers L1 and L2, the second marker A2 may not be placed on other layers. The second position 600 is located on the third layer L3 of the multi-layer sample rack 7, and is the position where the first marker A1 is expected to reach when the sample shovel 2 retrieves a sample from the third layer L3. When the first marker A1 is located at this second position 600, the sample shovel 2 can accurately retrieve the object to be moved and the sample placed on it. Although a specific example of the third layer L3 is shown in Figure 15, the third layer L3 is not limited to this one. Here, "third layer L3" refers to any layer of the sample holder 7 other than the first layer L1 and the second layer L2. It should be noted that this example illustrates the third layer L3 located between the first layer L1 and the second layer L2, but the relationship between the three layers is not limited to this. Alternatively, the third layer L3 could be located above or below the first layer L1 and the second layer L2, and the vertical relationship between the first layer L1 and the second layer L2 could also be reversed. Furthermore, it should be noted that the terms "first," "second," and "third" here are not intended to be sequential; there is no order between the first layer L1, the second layer L2, and the third layer L3, but are used for illustrative purposes. Additionally, since the dimensions of the sample holder 7 and the positions or spacing between each layer are known, the positional information between each layer can be predetermined and stored. Here, "positional information" refers to the relative positional information between layers, or the specific position of each layer.

[0199] First, the process of taking samples from the first layer L1 of the sample holder 7 will be explained. The process is similar for the second layer L2.

[0200] Since the first target position 100 is located in the first layer L1 and has been calibrated, it is only necessary to control the robotic arm 3 to move based on the relative position information between the origin position 00 and the first position 500, which is predetermined or pre-stored, and the first control information recorded in the first information recording step, so that the first marker A1 can be moved from the first position 500 to the first updated position 300.

[0201] Next, the process of taking samples from the third layer L3 of sample holder 7 will be explained.

[0202] First, the second position 600 is determined based on the first updated position 300, the second updated position 400, and pre-stored positional information related to the third layer L3, the first layer L1, and the second layer L2. Specifically, for example, knowing the above three pieces of information, the second position 600 can be determined by linear interpolation. However, the method for determining the second position 600 is not limited to linear interpolation, and other suitable methods can also be used.

[0203] After determining the second position 600, the fourth movement step is executed. Specifically, in the fourth movement step, based on the first control information recorded in the first information recording step, the second control information recorded in the second information recording step, the determined position information of the second position 600, and the pre-stored relative position information between the first position 500 and the origin position 00, the robotic arm 3 is controlled to move so that the first marker A1 moves from the first position 500 to the second position 600. More specifically, firstly, based on the first control information, the second control information, and the determined position information of the second position 600, control information corresponding to the movement of the first marker A1 from the origin position 00 to the second position 600 is determined, referred to as the third control information. Then, based on the determined third control information and the relative position information between the first position 500 and the origin position 00, the robotic arm 3 is controlled to move so that the first marker A1 moves from the first position 500 to the second position 600. After the first marker A1 moves to the second position 600, the sample shovel 2 can accurately pick up the object to be moved located in the third layer L3.

[0204] Furthermore, it should be noted that the control methods of the above-described embodiments and their variations in this application can also be implemented as computer software programs. For example, one embodiment of this application includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods shown in the figures.

[0205] Furthermore, it should be understood that although various embodiments and variations thereof of this application have been described with reference to an example of a sample moving device in the accompanying drawings, it should be understood that the embodiments within the scope of this application can be applied to other scientific instruments with similar structures and / or functions.

[0206] The foregoing description has already given many features and advantages, including various alternative implementations, as well as details of the structure and function of the apparatus and methods. This document is intended to be exemplary and is not exhaustive or limiting.

[0207] It will be apparent to those skilled in the art that various modifications can be made within the full scope indicated by the broad superordinate meaning of the terms expressed in the appended claims, particularly in terms of structure, materials, elements, components, shapes, dimensions, and arrangements of components, including combinations of these aspects within the scope of the principles described herein. Such various modifications are intended to be included herein, provided they do not depart from the spirit and scope of the appended claims.

[0208] Symbol Explanation: S Sample Movement System O Sample Movement Device 1 Camera 2 Sample Shovel (Actuator) 3 Robotic Arm 4 Rotary Tower (Robotic Arm Support) 5 Base 6 Calibration Pan 7 Sample Holder 8 Injector A1 First Marker A2 Second Marker L1 First Layer L2 Second Layer L3 Third Layer 00 Origin Position 100 First Target Position 200 Second Target Position 300 First Updated Position 400 Second Updated Position 500 First Position 600 Second Position

Claims

1. A control method for a sample moving device, the sample moving device comprising a camera, a robotic arm, and an actuator connected to the robotic arm, the actuator being used to enable movement of a moving object used to place a sample, characterized in that, The control method includes: In the acquisition step, the camera acquires an image that simultaneously contains the current positions of a first marker and a second marker, wherein the first marker is arranged to be fixed relative to the actuator and the position of the second marker is arranged to be fixed relative to the target position. Determine the steps, In the determination step, based on the positional information between the first marker and the second marker in the image, and the pre-stored relative positional information between the first marker and the second marker when the first marker is located at the target position, the difference between the current position of the first marker and the target position is determined, and In the determining step, the amount of the first marker to be moved is determined based on the difference; In the first movement step, the robotic arm is controlled based on the amount to be moved to move the first marker from its current position to the updated position; and The judgment step involves determining whether the difference between the updated position and the target position is less than or equal to a preset threshold. If the difference is greater than the threshold, the acquisition step is then executed.

2. The control method for a sample moving device as described in claim 1, characterized in that, The location-related information is the relative position information between the first marker and the second marker in the image.

3. The control method for a sample moving device as described in claim 1, characterized in that, The difference includes difference components across multiple degrees of freedom.

4. The control method for a sample moving device as described in claim 3, characterized in that, The determining step further includes: - Determine the degree of freedom that satisfies the preset conditions among the plurality of degrees of freedom as the determined degree of freedom, and determine the amount of movement to be made of the first marker based on the difference component of the difference in the determined degree of freedom.

5. The control method for a sample moving device as described in claim 4, characterized in that, The determination step also includes: - Determine whether the condition that the difference between the updated position and the target position on each of the plurality of degrees of freedom is less than or equal to the component of the threshold on that degree of freedom is met. If the condition is not met, then execute the acquisition step.

6. The control method for a sample moving device as described in claim 4, characterized in that, The degrees of freedom are translational degrees of freedom, and the determination step further includes... - Determine the one with the largest difference among the multiple translational degrees of freedom as the determined degree of freedom.

7. The control method for a sample moving device as described in claim 4, characterized in that, The sample moving device also includes a motor for driving the robotic arm to move, wherein the degree of freedom is translational degree of freedom. The control method includes a second movement step before the first execution of the acquisition step. In the second movement step, the robotic arm is controlled to move the first marker from the origin position to an initial position, where the origin position is a fixed position relative to the sample moving device, and the initial position is the current position of the first marker before the first movement step is executed for the first time. The control method further includes a recording step after the judgment step, the recording step including: - Record the change in motor parameters of the motor corresponding to the first marker from the origin position to the updated position.

8. The control method for a sample moving device as described in claim 7, characterized in that, The recording step is followed by a third movement step, in which the actuator is moved by controlling the robotic arm based on the change in the motor parameters.

9. The control method for a sample moving device as described in claim 1, characterized in that, A plurality of second markers are fixed relative to the target position, and the plurality of second markers are located on opposite sides of the target position.

10. The control method for a sample moving device as described in claim 1, characterized in that, The control method includes, in sequence: The first stage includes the acquisition step, the determination step, the first movement step, and the judgment step, wherein the target location is a first target location. The first stage further includes a second movement step before the first execution of the acquisition step. In the second movement step, the robotic arm is controlled to move the first marker from the origin position to an initial position. The origin position is a fixed position relative to the sample moving device, and the initial position is the current position of the first marker before the first movement step is executed for the first time in the first stage. The first stage judgment step also includes - First information recording step, in which first control information corresponding to the displacement of the first marker from the origin position to the updated position is recorded; Move the first marker back from the updated position to the original position; and The second stage includes the second movement step, the acquisition step, the determination step, the first movement step, and the judgment step. The target location is a second target location different from the first target location. The judgment step in the second stage is followed by... - Second information recording step, in which second control information corresponding to the displacement of the first marker from the origin position to the updated position is recorded.

11. The control method for a sample moving device as described in claim 10, characterized in that, After the second stage, the control method further includes - Control the robotic arm based on the first control information and the second control information, so that the first marker moves from the updated position in the first information recording step to the updated position in the second information recording step, or moves from the updated position in the second information recording step to the updated position in the first information recording step.

12. The control method for a sample moving device as described in claim 10, characterized in that, The first target location and its corresponding second marker in the first stage are located in the first layer of the multi-layer sample holder. The second target location and its corresponding second marker in the second stage are located in the second layer of the multi-layer sample holder. After the second stage, the control method further includes: - A second position is determined based on the first updated position, the second updated position, and pre-stored positional information between the third layer and the first and second layers in a multi-layered system. The first updated position is the updated position in the first information recording step, the second updated position is the updated position in the second information recording step, and the second position is the position in the third layer where the first marker is to be reached; and - Fourth movement step, in which the robotic arm is controlled based on the first control information, the second control information, the second position, and the pre-stored relative position information between the first position and the origin position, so that the first marker moves from the first position to the second position, where the first position is the position of the first marker before the fourth movement step is executed.

13. A sample moving device, comprising: A camera, used to capture images to form an image that simultaneously includes the current position of a first marker and the current position of a second marker; An actuator for enabling a movable object used to place a sample to move; as well as A robotic arm, wherein the robotic arm is connected to the actuator. Its features are, The first marker is positioned at a fixed location relative to the actuator, and the second marker is positioned at a fixed location relative to the target location. The sample moving device further includes: The acquisition module is configured to acquire the image; The first determining module is configured to determine the difference between the current position of the first marker and the target position based on the position-related information between the first marker and the second marker in the image obtained by the acquisition module, and the pre-stored relative position information between the first marker and the second marker when the first marker is located at the target position. A second determining module is configured to determine the amount of the first marker to be moved based on the difference determined by the first determining module. A control module, configured to control the robotic arm based on the amount of movement to be determined by the second determining module, to move the first marker from the current position to the updated position; and The judgment module is configured to determine whether the difference between the updated position and the target position is less than or equal to a preset threshold. If it is greater than the threshold, the acquisition module, the first determination module, the second determination module, and the control module are repeatedly executed until it is determined to be less than or equal to the threshold.

14. The sample moving device as described in claim 13, characterized in that, The robotic arm is configured to move in multiple degrees of freedom. The second determining module is configured to determine the degree of freedom that satisfies a preset condition among the plurality of degrees of freedom as the determined degree of freedom, and to determine the amount of movement to be made of the first marker based on the difference component of the difference in the determined degree of freedom.

15. The sample moving device as described in claim 13, characterized in that, The robotic arm is one of the following: multi-joint robotic arm, Cartesian coordinate robotic arm, cylindrical coordinate robotic arm, polar coordinate robotic arm, parallel robotic arm, SCARA robotic arm, or collaborative robotic arm.

16. The sample moving device as described in claim 14, characterized in that, The sample moving device also includes a motor connected to the robotic arm, which drives the robotic arm to move in the multiple degrees of freedom based on the amount of the first marker to be moved.

17. The sample moving device according to any one of claims 14 to 16, characterized in that, The degrees of freedom mentioned are translational degrees of freedom. The determined degree of freedom is the translational degree of freedom with the largest difference component among the multiple translational degrees of freedom.

18. The sample moving device according to any one of claims 13 to 16, characterized in that, A plurality of second markers are fixed relative to the target position, and the plurality of second markers are located on opposite sides of the target position.

19. A sample moving system, characterized in that, include: The sample moving device according to any one of claims 13 to 18; as well as A calibration disk, the calibration disk including the second marker, and the calibration disk being arranged at a fixed position relative to the target position.

20. The sample moving system as described in claim 18, characterized in that, The first marker is disposed on the actuator.

21. The sample moving system as described in claim 19 or 20, characterized in that, The sample moving system also includes a sample holder, on which the calibration disk is placed.

22. A computer-readable storage medium storing a computer program, characterized in that, The computer program is executed by a processor to implement the control method according to any one of claims 1 to 12.

23. A computer program product, comprising a computer program, characterized in that, The computer program is executed by a processor to implement the control method according to any one of claims 1 to 12.

Images