Method, device and medium for automatic calibration of internal module space position of instrument based on binocular vision

Abstract

The application relates to a binocular vision-based automatic calibration method for the spatial position of an internal module of an instrument, and comprises the following steps: acquiring the positions of cameras installed in different directions of the instrument; acquiring selected standard objects; under the condition of the same imaging resolution, performing coarse adjustment by using a large field of view, and then performing fine adjustment by reducing the field of view; acquiring a set initial step length; controlling a mechanical arm to move according to the step length; identifying the image boundary and comparing the consistency of pixel positions; judging whether the position deviation of the standard object and a probe is less than a set error limit; if yes, the current position is acquired; otherwise, the step length is updated, and the step of controlling the mechanical arm to move according to the step length is jumped to. The application is applied to the calibration process of a three-dimensional mechanical arm of an instrument, and computer vision technology is used to assist manual operation, so that the calibration process is automatically realized, the calibration steps are simplified, the requirement for an operator is reduced, and the standardization of the calibration process and the calibration result is easy to realize.

Description

Technical Field

[0001] This invention relates to the field of automated instrument technology, and in particular to a method, device and medium for automatic calibration of the spatial position of internal modules of an instrument based on binocular vision. Background Technology

[0002] During instrument position calibration, a robotic arm (actuator) is typically manually controlled to guide the probe (such as a pipette needle) to the target position. The target position is then visually confirmed, and the position information from the built-in position sensor is set as the instrument parameters. This process has low requirements for instrument functionality, is simple to implement, and easy to operate, but its accuracy is easily affected by human operation.

[0003] Similar applications exist in the calibration process of coordinate measuring instruments. Generally, the probe is manually guided close to the calibration tool (standard ball), and the tip of the probe contacts the measurement part of the part being measured in a point-contact manner. At the same time, the sensor on the probe generates a contact signal, and the instrument determines the position of the probe and the contact point through the data from the built-in position sensor.

[0004] This method requires initial manual guidance. During the approach process, the probe can detect minute deformations during point contact between the probe and the object being tested. At the same time, the actuator needs to be able to respond to the contact signal in real time. Summary of the Invention

[0005] In order to achieve the above-mentioned objectives and other advantages of the present invention, a first objective of the present invention is to provide an automatic calibration method for the spatial position of an instrument's internal modules based on binocular vision, comprising the following steps:

[0006] Obtain the positions of cameras installed in different directions on the instrument;

[0007] Obtain the selected standard;

[0008] Under the same imaging resolution, a large field of view is used for coarse adjustment, and then the field of view is reduced for fine adjustment.

[0009] Get the initial step size set;

[0010] Control the robotic arm to move according to the step length;

[0011] Image boundaries are identified, and the consistency of pixel positions is compared.

[0012] Determine whether the positional deviation between the standard object and the probe is less than the set error limit;

[0013] If so, get the current position;

[0014] Otherwise, update the step size and jump to the step where the robotic arm moves according to the step size.

[0015] Furthermore, the camera's optical axis is mounted non-parallel in two directions on the instrument.

[0016] Furthermore, the camera is mounted in two directions of the instrument in an orthogonal manner along its optical axis.

[0017] Furthermore, it also includes the steps of: adjusting the camera position, determining whether the standard object is located in the central area of ​​the image, stopping the adjustment of the camera position if yes, and continuing to adjust the camera position otherwise.

[0018] Furthermore, it also includes the steps of calibrating the distortion of the binocular camera and calibrating the three-dimensional image.

[0019] Furthermore, the process of coarse adjustment using a large field of view and fine adjustment by narrowing the field of view under the same imaging resolution includes the following steps:

[0020] Adjust the object distance and focal length of the zoom lens to determine whether the field of view covers the entire instrument.

[0021] If so, adjust the object distance and focal length of the zoom lens, reduce the field of view to half of the initial field of view, and adjust the position;

[0022] Repeat the above adjustment steps until the position parameters before and after adjustment are less than the set value.

[0023] Furthermore, the initial step size is set to 5% of the total stroke.

[0024] Furthermore, the process of identifying image boundaries and comparing the consistency of pixel positions includes the following steps:

[0025] In the process of image boundary recognition, the rectangular regions of objects in the image are identified. Gaussian filtering is used to eliminate noise, and the Sobel operator is used to extract the boundaries in the x and y directions and extract the vertices of the rectangle.

[0026] Sub-pixel-level positioning is achieved by determining the vertex of the cone-shaped probe at the boundary intersection.

[0027] A second objective of the present invention is to provide an electronic device comprising: a memory having program code stored thereon; a processor connected to the memory, and wherein, when the program code is executed by the processor, an automatic calibration method for the spatial position of an instrument's internal modules based on binocular vision is implemented.

[0028] A third objective of this invention is to provide a computer-readable storage medium having program instructions stored thereon, which, when executed, implement an automatic spatial position calibration method for internal modules of an instrument based on binocular vision.

[0029] Compared with the prior art, the beneficial effects of the present invention are:

[0030] This invention provides an automatic calibration method for the spatial position of internal modules of an instrument based on binocular vision. It is applied to the calibration process of a three-dimensional robotic arm for an instrument. By using computer vision technology to assist manual operation, the calibration process is automated, the calibration steps are simplified, the requirements for operators are reduced, and the calibration process and results are easily standardized.

[0031] Compared to coordinate measuring machines (CMMs), this invention has lower requirements for instrument hardware functionality. It uses external image data instead of internal sensor signal feedback, making it more versatile, requiring less hardware, and easier to optimize performance and expand functionality. Furthermore, compared to traditional methods, it achieves faster convergence in the coarse-tuning stage. It can also be used in conjunction with traditional methods, employing the computer vision-assisted technology of this invention for coarse-tuning in the early stages and traditional methods for fine-tuning in the later stages.

[0032] This invention has a high degree of automation and can be performed by end users for automatic instrument maintenance.

[0033] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it according to the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Specific embodiments of the present invention are given in detail below with reference to the accompanying drawings. Attached Figure Description

[0034] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0035] Figure 1 This is a flowchart of the automatic spatial position calibration method for the internal module of an instrument based on binocular vision, as described in Example 1.

[0036] Figure 2 This is a schematic diagram of the camera installation in Example 1;

[0037] Figure 3 This is a schematic diagram of the electronic device in Example 2;

[0038] Figure 4 This is a schematic diagram of a computer-readable storage medium according to Example 3. Detailed Implementation

[0039] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments. It should be noted that, without conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0040] Example 1

[0041] Automatic calibration method for the spatial position of internal modules of instruments based on binocular vision, such as Figure 1 As shown, it includes the following steps:

[0042] Obtain the positions of cameras mounted in different directions on the instrument; such as Figure 2 As shown, the optical axes of the two cameras are mounted in two directions on opposite sides of the instrument without being parallel. Preferably, the cameras are mounted in two directions on opposite sides of the instrument with their optical axes orthogonal, such as in front and to the left (or right).

[0043] Obtain the selected standard object, such as a reference object on the instrument base.

[0044] The robotic arm is manually guided to bring the probe into the camera's field of view.

[0045] Under the same imaging resolution, a large field of view is used for coarse adjustment, and then the field of view is reduced for fine adjustment; appropriately reducing the camera's field of view is beneficial to improving the final adjustment accuracy. Specifically, it includes the following steps:

[0046] Using a zoom lens, initially adjust the object distance and focal length of the zoom lens to ensure the field of view covers the entire instrument, and perform a coarse position adjustment;

[0047] After completing the coarse adjustment, adjust the object distance and focal length of the zoom lens, reduce the field of view to half of the initial field of view, and make position adjustments;

[0048] Repeat the above adjustment steps until the position parameters before and after adjustment are less than the set value.

[0049] Obtain the initial step size; due to differences in instrument actuators, the initial step size generally needs to be set manually, such as setting the initial step size to 5% of the total stroke. Since it is assumed that the actuator does not have a contact signal response function, a smaller step size is generally selected.

[0050] Control the robotic arm to move according to the step length;

[0051] Image recognition identifies image boundaries and compares the consistency of pixel positions, thus eliminating the need for distortion correction. Specifically, it includes the following steps:

[0052] In the process of image boundary recognition, the rectangular regions of objects in the image are identified. Gaussian filtering is used to eliminate noise, and the Sobel operator is used to extract the boundaries in the x and y directions, thereby extracting the vertices of the rectangle.

[0053] The probe uses a conical probe. After noise is eliminated, the two side boundaries are extracted, and the vertex of the conical probe is determined through the intersection of the boundaries to achieve sub-pixel level positioning.

[0054] Determine whether the positional deviation between the standard object and the probe is less than the set error limit;

[0055] If so, get the current position;

[0056] Otherwise, update the step size and jump to the step size control procedure to move the robotic arm according to the step size.

[0057] Since the distortion is smaller in the central region of the camera, adjust the camera position and determine if the standard object is located in the center of the image. If it is, stop adjusting the camera position; otherwise, continue adjusting the camera position. Placing the standard object in the center of the image is beneficial for final accuracy adjustments.

[0058] Pre-calibration of the binocular camera distortion and the calibration of the 3D images allow for more effective adjustment of the step size during the calibration process. Specifically, this includes the following steps:

[0059] The pixel coordinate systems of the two images are transformed to the camera coordinate system using a common intrinsic parameter matrix:

[0060] A new camera coordinate system is obtained by rotating the two camera coordinate systems respectively;

[0061] Perform distortion correction operations on both cameras separately for the new camera coordinate system;

[0062] After the distortion correction operation is completed, the two camera coordinate systems are transformed back to the two image pixel coordinate systems using the intrinsic parameter matrices of the two cameras.

[0063] The pixel values ​​of the new image are interpolated using the pixel values ​​of the two source images respectively.

[0064] Example 2

[0065] An electronic device 200, such as Figure 3 As shown, the system includes, but is not limited to: a memory 201 storing program code; and a processor 202 connected to the memory, which, when executed by the processor, implements an automatic spatial position calibration method for internal instrument modules based on binocular vision. For a detailed description of the method, please refer to the corresponding description in the above method embodiments, which will not be repeated here.

[0066] Example 3

[0067] A computer-readable storage medium, such as Figure 4 As shown, it stores program instructions, which, when executed, implement an automatic spatial position calibration method for the internal modules of the instrument based on binocular vision. For a detailed description of the method, please refer to the corresponding description in the above method embodiments, which will not be repeated here.

[0068] Example 4

[0069] A computer program product includes a computer program / instructions that, when executed by a processor, implement an automatic spatial position calibration method for internal instrument modules based on binocular vision. A detailed description of the method can be found in the corresponding descriptions in the above method embodiments, and will not be repeated here.

[0070] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0071] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0072] The above are merely embodiments of this specification and are not intended to limit the scope of one or more embodiments of this specification. For those skilled in the art, various modifications and variations can be made to one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of one or more embodiments of this specification.

Claims

1. A method for automatic calibration of the spatial position of an internal module of an instrument based on binocular vision, characterized in that, Includes the following steps: Obtain the positions of cameras installed in different directions on the instrument; Obtain the selected standard; Under the same imaging resolution, a large field of view is used for coarse adjustment, and then the field of view is reduced for fine adjustment. Get the initial step size set; Control the robotic arm to move according to the step length; Image boundaries are identified, and the consistency of pixel positions is compared. Determine whether the positional deviation between the standard object and the probe is less than the set error limit; If so, get the current position; Otherwise, update the step size and jump to the step where the robotic arm moves according to the step size; The process of coarse adjustment using a large field of view and fine adjustment by narrowing the field of view under the same imaging resolution includes the following steps: Adjust the object distance and focal length of the zoom lens to determine whether the field of view covers the entire instrument. If so, adjust the object distance and focal length of the zoom lens, reduce the field of view to half of the initial field of view, and adjust the position; Repeat the above adjustment steps until the position parameters before and after adjustment are less than the set value; The process of identifying image boundaries and comparing the consistency of pixel positions includes the following steps: In the process of image boundary recognition, the rectangular regions of objects in the image are identified. Gaussian filtering is used to eliminate noise, and the Sobel operator is used to extract the boundaries in the x and y directions and extract the vertices of the rectangle. Sub-pixel-level positioning is achieved by determining the vertex of the cone-shaped probe at the boundary intersection.

2. The binocular vision based instrument internal module space position automatic calibration method according to claim 1, characterized in that: The camera's optical axis is mounted in two directions on the instrument without parallel alignment.

3. The binocular vision based instrument internal module space position automatic calibration method according to claim 2, characterized in that: The camera is mounted in two directions on the instrument with its optical axis orthogonal.

4. The binocular vision based instrument internal module space position automatic calibration method according to claim 1, wherein, It also includes the following steps: adjust the camera position, determine whether the standard object is located in the central area of ​​the image, if so, stop adjusting the camera position, otherwise continue adjusting the camera position.

5. The automatic spatial position calibration method for internal instrument modules based on binocular vision according to claim 1, characterized in that, It also includes the steps of calibrating the distortion of the binocular camera and calibrating the 3D image.

6. The automatic spatial position calibration method for internal instrument modules based on binocular vision according to claim 1, characterized in that: The initial step size is set to 5% of the total travel.

7. An electronic device, characterized in that, include: A memory that stores program code; A processor, which is connected to the memory, and which, when the program code is executed by the processor, implements the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, It stores program instructions that, when executed, implement the method as described in any one of claims 1 to 6.

Images