Test tube positioning method and device based on auxiliary mark

Abstract

The invention relates to a test tube positioning method and device based on an auxiliary mark. The method comprises the steps that calibration of internal parameters of a camera is completed, imaging distortion correction is conducted through the calibrated internal parameters, and the parameter sizes of a test tube and a test tube rack are determined. And performing id identification and feature angular point extraction on the square reference mark, solving the mark pose through a PnP algorithm, and determining the relative reference pose of the test tube rack. And solving a test tube orifice circle contour plane through homography matrix mapping, solving an ideal circle center coordinate based on a contour circle extraction algorithm, and taking the ideal circle center coordinate as a test tube index parameter. And identifying the tube orifice circle contour of the target test tube, calculating the Euclidean distance between the detected circle center coordinate and the test tube index parameter, and judging whether the test tube exists or not according to the distance. By introducing a square reference mark and a homography matrix mapping technology, points on an image plane can be accurately mapped to a test tube orifice circular contour plane. The problem of false detection caused by light source change is avoided, and the positioning stability and reliability are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of automated test tube transfer and sorting, and particularly to a test tube positioning method and device based on auxiliary marking. Background Art

[0002] Test tubes are common instruments in medical and chemical laboratories. They are usually round-bottomed cylinders, open at one end and closed at the other end, and are mostly made of glass. They are used to hold a small amount of liquid or solid reagents, heat a small amount of solid or liquid, or collect a small amount of gas, etc. Since test tubes are often used to hold test reagents to be tested, to avoid environmental pollution, test tubes are usually equipped with sealing sleeves or lids of obvious colors. Since test tubes are usually made of transparent glass, it is difficult to directly locate them visually. Therefore, at present, they are often identified and located by the color of the tube mouth of the test tube. Currently, the mainstream test tube positioning and identification methods on the market include time-domain color detection, deep learning detection, and feature contour detection, etc.

[0003] In the time-domain color detection technology, by performing pixel difference on different images under time-domain changes, the area where the pixels change can be obtained. Under the condition of a constant light source, this method can relatively accurately obtain the area information of the test tube changes. However, when the light source changes over time, the differential operation will also consider the pixel differences caused by overexposure or underexposure changes of the light source, which will cause a certain degree of misjudgment in the target area. In addition, after determining the target area of the test tube, a comparison is needed to find the corresponding target test tube index. This technology obtains the ROI area of the test tube rack through contour detection, and then obtains the ideal position index of the test tube. However, the edge contour of the test tube rack belongs to weak texture features, with poor robustness and is easily affected by the environment, resulting in a large deviation in the extracted area. Moreover, considering the generally small structural size of the test tube rack, this will cause error accumulation, and when performing index matching in the recognition area, it is easy to match the adjacent test tube index, with a certain possibility of misjudgment.

[0004] In the deep learning detection technology, in the above-mentioned technology, the semantic segmentation network based on deep learning for identifying and positioning test tubes on a medical tray requires continuous adjustment of parameters with a training dataset, and it is difficult to be quickly deployed and applied in different scenarios, with a relatively high training and deployment cost. In the feature contour detection technology, the existing methods need to add additional labels to the test tubes, which may not only interfere with the observation of the reagent state, but also once the labels are damaged, fallen off or contaminated, the accuracy of label recognition will be greatly reduced, thereby affecting the verification effect of the number of test tubes. Summary of the Invention

[0005] The first object of the present invention is to provide a test tube positioning method based on auxiliary marking, which aims to solve the technical problems of inaccurate test tube positioning, large interference from the environment, and dependence on complex datasets in the existing test tube positioning and identification methods.

[0006] To solve the above technical problems, a test tube positioning method based on auxiliary marking is provided, including:

[0007] S1. Calibrate the internal parameters of the camera, correct the imaging distortion using the calibrated internal parameters, and clarify the parameter sizes of the test tube and the test tube rack;

[0008] S2. Perform id recognition and extract feature corner points for the square reference mark, and solve the mark pose through the PnP algorithm to determine the relative reference pose of the test tube rack;

[0009] S3. Solve the plane of the test tube nozzle circular contour by mapping with a homography matrix, solve the ideal center coordinates based on the contour circle extraction algorithm, and use them as test tube index parameters;

[0010] S4. Identify the circular contour of the nozzle of the target test tube, calculate the Euclidean distance between the detected center coordinates and the test tube index parameters, and judge whether the test tube exists based on this distance;

[0011] S5. Calculate the spatial pose of the test tube in the camera coordinate system, and then perform a three-dimensional simulation presentation of the overall physical system.

[0012] Further, the step S1 includes:

[0013] S11. Use the camera to take multiple plane target images at different angles, and perform camera calibration using the Zhang's calibration method to obtain the camera internal parameter matrix and the distortion coefficient;

[0014] S12. Complete the distortion correction of the acquired images based on the calibrated camera internal parameter matrix and the distortion coefficient;

[0015] S13. Determine the id and overall size of the square reference mark, fix the square reference mark in the blank area on the upper surface of the test tube rack, use the upper left corner coordinates of the square reference mark as the reference point of the test tube rack, set the camera coordinate system, select the specification size of the test tube, and at the same time set the vertical distance between the center of the test tube nozzle circle and the upper surface of the test tube rack as h t , and the distance between the centers of the adjacent test tube nozzles is x d .

[0016] Further, the step S2 includes:

[0017] S21. Obtain the id of the mark and the pixel coordinates of the four outer corner points through the mark recognition algorithm;

[0018] S22. Combine the known mark size, use the PnP algorithm to solve the rotation matrix and translation vector of the mark relative to the camera coordinate system, and use this pose as the pose of the test tube rack relative to the camera reference.

[0019] Further, the step S3 includes:

[0020] S31. Calculate the rotation matrix and translation vector of the test tube nozzle circular contour plane relative to the camera coordinate system according to the pose relationship between the reference coordinate system of the test tube nozzle circular contour plane and the camera coordinate system;

[0021] S32. Map the image plane to the test tube nozzle circular contour plane by using the homography matrix;

[0022] S33. Solve the ideal center coordinates based on the contour circle extraction algorithm, and use the ideal center coordinates as the test tube index parameter.

[0023] Further, the step S4 includes:

[0024] S41. Perform circular contour detection on the test tube nozzle circular contour plane image to obtain all possible circular contours in the image;

[0025] S42. Filter out abnormal contours according to the threshold constraint to obtain the remaining test tube nozzle circular contours;

[0026] S43. Calculate the Euclidean distance between the detected center coordinates and the test tube index parameter;

[0027] S44. When the distance meets the set threshold condition, it is determined that there is a test tube at this position;

[0028] S45. Obtain the storage situation of the test tubes in this scenario by tracing the index sequence of the circular contour on the test tube rack.

[0029] Further, the Euclidean distance judgment condition formula is:

[0030]

[0031] Among them, is the center coordinate of the detected test tube nozzle contour circle, and P i ′(u i ′, v i ′) is the pixel coordinate of the center of the test tube nozzle circle in the ideal state, and r j is the radius of the contour circle.

[0032] Further, the calculation formula of the homography matrix H is:

[0033] H = K[r1 r2 t]

[0034] Among them, K is the camera internal parameter matrix, r1 and r2 are the first two columns of the rotation matrix, and t is the translation vector.

[0035] The second object of the present invention is to provide a test tube positioning device based on auxiliary markings, and the device includes:

[0036] A test tube rack, which is formed with a plurality of placement slots for placing test tubes;

[0037] A square reference marking, which is arranged beside the placement slot;

[0038] A camera, which is used to capture relevant image information of the square reference marking on the test tube rack and the test tubes placed on the test tube rack.

[0039] Further, the square reference marking includes an ArUco marking, an AprilTag marking or an ARTag marking.

[0040] Implementing the embodiments of the present invention will have the following beneficial effects:

[0041] In the test tube positioning method based on auxiliary markings in this embodiment, by introducing the square reference marking and the homography matrix mapping technology, the points on the image plane can be accurately mapped to the circular contour plane of the test tube mouth. This mapping relationship enables the system to accurately identify the position and posture of the test tube in a complex environment. Even when the relative position between the test tube rack and the camera changes, a high-precision positioning effect can be maintained. Compared with the traditional time-domain color detection technology, the present invention effectively avoids the problem of false detection caused by light source changes, and significantly improves the stability and reliability of positioning.

[0042] The present invention has no specific requirements for the placement position of the camera. Even when the relative position between the test tube rack and the camera changes, the reliability of test tube identification can be ensured. This solves the problem that the position of the camera cannot be flexibly moved in the time-domain color detection technology, making the present invention have a wider range of applicability. In addition, the present invention is based on a feature recognition algorithm and does not require complex dataset training. Only by setting the test tube size parameters, etc., it can be put into use, and has a wider market application value. Description of the Drawings

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0044] Figure 1 It is a flowchart of the test tube positioning method based on auxiliary markings according to the embodiments of the present invention;

[0045] Figure 2 It is a flowchart of the method of step S1 according to the embodiments of the present invention;

[0046] Figure 3 It is the flowchart of the method for step S2 in the embodiments of the present invention;

[0047] Figure 4 It is the flowchart of the method for step S3 in the embodiments of the present invention;

[0048] Figure 5 It is the flowchart of the method for step S4 in the embodiments of the present invention;

[0049] Figure 6 It is the schematic diagram of the square reference mark in the embodiments of the present invention;

[0050] Figure 7 It is the schematic diagram of the extraction process of the mark pose in the embodiments of the present invention;

[0051] Figure 8 It is the schematic diagram of obtaining the circular plane of the tube orifice in the embodiments of the present invention;

[0052] Figure 9 It is the schematic diagram of the three-dimensional positioning of the test tube in the embodiments of the present invention;

[0053] Figure 10 It is the schematic diagram of the pose simulation effect in the embodiments of the present invention;

[0054] Figure 11 It is the schematic diagram of the structure of the test tube positioning device based on the auxiliary mark in the embodiments of the present invention.

[0055] Wherein: 100, the test tube positioning device based on the auxiliary mark; 110, the test tube rack; 111, the placement groove; 120, the square reference mark; 121, the characteristic corner point; 130, the camera; 140, the test tube index parameter; 150, the circular contour of the test tube orifice; 200, the test tube. Detailed implementation manners

[0056] To facilitate the understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant drawings. The preferred embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, these embodiments are provided to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0057] It should be noted that when an element is referred to as being "fixed to" another element, it can be directly on the other element or there may also be an intermediate element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there may be an intermediate element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0058] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this invention belongs. The terms used in the specification of this invention are for the purpose of describing specific embodiments only and are not intended to limit the invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0059] Please refer to Figures 1-10 , an embodiment of the present invention provides a method for positioning a test tube 200 based on an auxiliary marker, including:

[0060] S1. Calibrate the internal parameters of the camera 130, correct the imaging distortion using the calibrated internal parameters, and clarify the parameter sizes of the test tube 200 and the test tube rack 110; Exemplarily, the calibration of the camera 130 refers to the process of determining the internal parameters and external parameters of the camera 130. The internal parameters of the camera 130 include the internal parameter matrix and the distortion coefficient, and the external parameters include the position and attitude of the camera 130 in three-dimensional space. The calibration of the camera 130 generally uses a planar or stereo target to establish the mapping relationship between the three-dimensional coordinates of the feature points in space and the two-dimensional coordinates of their projection points, and then completes the solution of the internal and external parameters of the camera 130.

[0061] S2. Perform id recognition on the square reference marker 120 and extract the feature corner points 121, and solve the marker pose through the PnP algorithm to determine the relative reference pose of the test tube rack 110; Exemplarily, the square reference marker 120 is a binary square marker for target pose estimation, which consists of a black border and an internal binary matrix. The arrangement of this matrix determines its marker index (id). The black border helps to quickly detect the marker in the image, and the binary coding structure facilitates its identification as well as error detection and correction. The four outer feature corner points 121 of the marker are often used to achieve pose positioning and recognition, as Figure 6 shown.

[0062] S3. Solve the plane of the circular contour 150 of the test tube mouth by mapping with a homography matrix, solve the ideal center coordinates based on the contour circle extraction algorithm, and use them as the test tube index parameter 140; Exemplarily, the homography matrix is a matrix used to describe the projective transformation relationship between two planes. It can map the points on one plane to the corresponding points on another plane and is commonly used in fields such as image correction, stitching, and augmented reality.

[0063] S4. Identify the circular contour of the mouth of the target test tube 200, calculate the Euclidean distance between the detected center coordinates and the test tube index parameter 140, and determine whether the test tube 200 exists based on this distance;

[0064] S5. Solve the spatial pose of the test tube 200 in the coordinate system of the camera 130, and then perform a three-dimensional simulation presentation of the overall physical system.

[0065] In the test tube 200 positioning method based on auxiliary markers in this embodiment, by introducing a square reference marker 120 and a homography matrix mapping technique, points on the image plane can be accurately mapped to the plane of the test tube nozzle circular contour 150. This mapping relationship enables the system to accurately identify the position and orientation of the test tube 200 in a complex environment. Even when the relative position between the test tube rack 110 and the camera 130 changes, a high-precision positioning effect can be maintained. Compared with the traditional time-domain color detection technology, the present invention effectively avoids the problem of false detection caused by light source changes, and significantly improves the stability and reliability of positioning.

[0066] The test tube 200 positioning device and method described in this embodiment use the square reference marker 120 as a medium, and cleverly use the homography matrix to realize the mapping from the image plane to the plane of the test tube nozzle circular contour 150 and obtain the test tube nozzle circular contour 150 in the test tube nozzle contour plane through the nozzle circular contour algorithm. Through the Euclidean distance judgment condition, the accurate identification of the test tube 200 on the test tube rack 110 and the indexing of the test tube 200 are successfully realized, and the three-dimensional position of the test tube 200 is traced according to the obtained index. This method effectively solves the problem that the position of the test tube 200 cannot be visually detected at random placement positions in the current market, and has significant advantages such as flexible operation, strong generalizability, and wide adaptability.

[0067] The present invention has no specific requirements for the placement position of the camera 130. Even when the relative position between the test tube rack 110 and the camera 130 changes, the reliability of test tube 200 recognition can be ensured. This solves the problem that the position of the camera 130 cannot be flexibly moved in the time-domain color detection technology, making the present invention have a wider applicability. In addition, the present invention is based on a feature recognition algorithm and does not require complex dataset training. Only by setting the size parameters of the test tube 200, etc., it can be put into use, and has a wider market application value.

[0068] Please refer to Figure 2 , in a possible implementation manner, step S1 includes:

[0069] S11. Use the camera 130 to capture multiple plane target images at different angles, and perform calibration on the camera 130 using the Zhang's calibration method to obtain the camera internal parameter matrix and the distortion coefficients;

[0070] The present invention realizes the positioning and recognition of the test tube 200 based on image recognition technology. Therefore, it is first necessary to perform internal parameter calibration on the camera 130. Use the camera 130 to capture multiple plane target images at different angles, and perform calibration on the camera 130 using the Zhang's calibration method to obtain the camera internal parameter matrix and the distortion coefficients (k1, k2, p1, p2, k3). Among them, (f x , f yrespectively represent the normalized focal lengths of the camera 130 in the horizontal and vertical axes; (u0, v0) represent the principal point coordinates; k1, k2, and k3 respectively represent the first-order, second-order, and third-order radial distortion coefficients; p1 and p2 respectively represent the first-order and second-order tangential distortion coefficients. According to the obtained distortion coefficients, perform distortion correction processing on the image.

[0071] S12. Complete the distortion correction of the acquired image based on the calibrated camera internal parameter matrix and distortion coefficients; Lens distortion correction refers to, on the basis of completing the calibration of the camera 130, using the distortion coefficients and distortion model to remove the radial distortion and tangential distortion of the image and obtain an undistorted image.

[0072] Based on the camera internal parameter matrix K and distortion coefficients (k1, k2, p1, p2, k3) calibrated in step S11, complete the distortion correction of the acquired image: First, combine the normalized focal lengths (f x , f y ) and principal point coordinates (u0, v0) obtained by calibration in step S11, and convert the pixel coordinates of the distorted image to normalized image coordinates The specific normalization operation is as follows:

[0073]

[0074] Then, substitute the normalized image coordinates of the distorted image into the lens distortion model, and at the same time combine the distortion coefficients (k1, k2, p1, p2, k3) obtained by calibration in step 1.1 to calculate the undistorted normalized image coordinates (x, y) of the corresponding points. The specific lens distortion model is as follows:

[0075]

[0076] Among them, Finally, convert the calculated undistorted normalized image coordinates (x, y) to undistorted pixel coordinates (u, v) through an inverse normalization operation. The specific inverse normalization operation is as follows:

[0077]

[0078] After the original image is processed through the above steps, an undistorted image can be obtained.

[0079] S13. Determine the id and overall size of the square reference mark 120, fix the square reference mark 120 on the blank area of the upper surface of the test tube rack 110, use the upper left corner coordinates of the square reference mark 120 as the reference point of the test tube rack 110, set the camera 130 coordinate system, and select the specification size of the test tube 200. At the same time, set the vertical distance from the center of the tube mouth of the test tube 200 to the upper surface of the test tube rack 110 as ht , the center distance between the orifice centers of adjacent test tubes 200 is x d . Exemplarily, in the system device of the present invention, it is essential to clearly define key parameters. During the implementation of the invention, the feature of the square reference mark 120 is introduced into the device. Common square reference marks 120 include, but are not limited to, ArUco marks, AprilTag marks, and ARTag marks. The present invention selects the ArUco mark as the feature mark, determines the id and overall size of the mark, and then fixes it on the blank area of the upper surface of the test tube rack 110, ensuring that the mark is parallel to the orifice array of the test tube rack 110. Next, the upper left corner coordinates of the mark are used as the reference point O of the test tube rack 110 a -X a Y a Z a , set the coordinate system O of the camera 130 c -X c Y c Z c , and select the specification size of the test tube 200. At the same time, set the vertical distance between the center of the orifice circle of the test tube 200 and the upper surface of the test tube rack 110 to be h t , the center distance between the orifice centers of adjacent test tubes 200 is x d .

[0080] Please refer to Figure 3 , in a possible implementation manner, step S2 includes:

[0081] S21. Obtain the id of the mark and the pixel coordinates of the four outer corner points through the mark recognition algorithm;

[0082] S22. Combine the known mark size, and use the PnP algorithm to solve the rotation matrix and translation vector of the mark relative to the coordinate system of the camera 130, and use this pose as the pose of the test tube rack 110 relative to the reference of the camera 130. Exemplarily, place the camera 130 at a position where the entire image of the test tube rack 110 can be captured, and ensure that the mark on the test tube rack 110 is not blocked. Then, obtain the id of the mark and the pixel coordinates of the four outer corner points through the mark recognition algorithm, and combine the known mark size. Introduce the pixel coordinates of the four outer corner points as features into the PnP algorithm, and the rotation matrix of the mark relative to the coordinate system of the camera 130 can be solved and the translation vector and use this pose as the pose of the test tube rack 110 relative to the reference of the camera 130.

[0083] Please refer to Figure 4 , in a possible implementation manner, step S3 includes:

[0084] S31. Calculate the rotation matrix and translation vector of the plane of the circular contour 150 of the test tube mouth relative to the coordinate system of the camera 130 according to the pose relationship between the reference coordinate system of the plane of the circular contour 150 of the test tube mouth and the coordinate system of the camera 130;

[0085] Since the distance x from the center of the circular mouth of the test tube 200 to the upper surface of the test tube rack 110 is known d , regard the plane of the circular contour 150 of the test tube mouth as the target plane. Let the reference coordinate system of the plane of the circular contour 150 of the test tube mouth be O b -X b Y b Z b The pose relationship relative to the O a coordinate system is According to the principle of pose transformation, the rotation matrix of the reference coordinate system of the plane of the circular contour 150 of the test tube mouth relative to the coordinate system of the camera 130 is The translation vector is

[0086] To describe the mapping relationship between the image plane and the plane of the circular contour 150 of the test tube mouth, introduce the homography matrix H as an intermediate variable. Since the homography matrix H between planes can be jointly solved by and Assume that all points in the world coordinate system are located in the plane Z = 0, and its homogeneous coordinate is P w = [X Y 0 1] T , and the point P c in the coordinate system of the camera 130 is obtained by rigid body transformation:

[0087]

[0088] Among them, R [:,1:2] represents the first two columns of the rotation matrix, and t = [t x t y t z T represents the translation vector. The coordinate of the point P c = [X c Y c Z c T after being projected onto the normalized plane is Substitute the expression of P c to get:

[0089]

[0090] Among them, r ij is the element of the rotation matrix . Convert the normalized coordinates to pixel coordinates through the internal parameter matrix K:

[0091]

[0092] Express the projection process as a linear proportional relationship of homogeneous coordinates:

[0093]

[0094] where λ = r 31 X + r 32 Y + t z is the scale factor. After eliminating λ, the closed-form solution of the homography matrix H is:

[0095]

[0096] In summary, the homography matrix H between the image plane and the plane of the circular contour 150 of the test tube mouth is obtained. After performing homography matrix mapping on the image plane captured by the camera 130, the plane of the circular contour 150 of the test tube mouth can be obtained.

[0097] S32. Use the homography matrix to map the image plane to the plane of the circular contour 150 of the test tube mouth;

[0098] S33. Solve for the ideal center coordinates based on the contour circle extraction algorithm, and use the ideal center coordinates as the test tube index parameter 140.

[0099] Let the pose of the center of the circular mouth of the i-th test tube 200 under the reference of the test tube rack 110 be The pose in the coordinate system of the camera 130 can be obtained as According to the pinhole imaging model, the coordinates of the three-dimensional point projected onto the normalized image plane are where (x i ′, y i ′) are the normalized coordinates. The normalized coordinates are converted to image pixel coordinates through the intrinsic matrix K, and there is:

[0100]

[0101] The image pixel coordinates P i (u i , v i ) of the center of the circular mouth of the test tube 200 can be deduced through the above formula, According to the homography matrix H obtained above, there is Solve to get Finally, the pixel coordinates of the center of the circular mouth of the test tube 200 in the ideal state in the plane of the circular contour 150 of the test tube mouth are obtained, and these coordinates are used as the judgment basis for indexing the test tube 200.

[0102] Please refer to Figure 5 , in a possible implementation manner, step S4 includes:

[0103] S41. Perform circular contour detection on the planar image of the circular contour 150 of the test tube opening to obtain all possible circular contours within the image;

[0104] S42. Filter out abnormal contours according to threshold constraints to obtain the remaining circular contour 150 of the test tube opening;

[0105] S43. Calculate the Euclidean distance between the detected center coordinates of the circle and the test tube index parameter 140;

[0106] S44. When the distance meets the set threshold condition, it is determined that there is a test tube 200 at this position;

[0107] S45. By tracing the index sequence of the circular contour on the test tube rack 110, obtain the storage situation of the test tube 200 in this scenario.

[0108] After obtaining the planar image of the circular contour 150 of the test tube opening, perform circular contour detection on this plane to obtain all possible circular contours within the image, and filter out abnormal contours according to threshold constraints to obtain the remaining circular contour 150 of the test tube opening.

[0109] Let the center coordinates of the obtained contour circle be P j (u j , v j ), The radius of the contour circle is r j , and the center coordinates of the ideal contour circle are P i ′(u i ′, v i ′). Set the Euclidean distance judgment condition:

[0110]

[0111] When the above formula (10) is satisfied, it can be considered that there is a test tube 200 at this position, and by tracing the index sequence of the circular contour on the test tube rack 110, obtain the storage situation of the test tube 200 in this scenario, so as to determine which index position the existing test tube 200 is located in the test tube rack 110.

[0112] In a possible implementation manner, the Euclidean distance judgment condition formula is:

[0113]

[0114] Among them, P j (u j , v j ), is the center coordinates of the circular contour of the test tube 200 opening detected,

[0115] P i ′(u i′, v i ′) are the pixel coordinates of the center of the circle at the mouth of the test tube 200 in the ideal state, and r j is the radius of the contour circle.

[0116] Regarding step S5: The index of the target test tube 200 has been obtained in the above step S41. According to the index, the pose of the center of the circle at the mouth of the test tube 200 under the reference of the test tube rack 110 can be traced as And according to the pose transformation, the pose in the coordinate system of the camera 130 can be obtained as Among them

[0117] From the above steps, the pose of the test tube rack 110 relative to the camera 130 can be known The pose of the test tube 200 relative to the camera 130 can be known By modeling and constraining reasonable and accurate target reference points, the relationship among the three can be shown in the three-dimensional simulation situation.

[0118] In a possible implementation manner, the calculation formula of the homography matrix H is:

[0119] H = K[r1 r2 t]

[0120] Among them, K is the camera internal parameter matrix, which includes the focal length and principal point coordinates of the camera 130. It is used to convert points in the coordinate system of the camera 130 into image pixel coordinates. r1 and r2 are the first two columns of the rotation matrix. The first two columns of the rotation matrix R represent the rotation from the world coordinate system to the coordinate system of the camera 130, and t is the translation vector, representing the translation from the world coordinate system to the coordinate system of the camera 130..

[0121] In this implementation, the specification of the 2D planar checkerboard target used in the calibration process of the present invention is 12×9, and the side length of the grid is 20 mm. The camera 130 takes 20 images of the checkerboard target at different angles, and uses the "Camera Calibration Toolbox for Matlab" toolbox developed by Jean-Yves Bouguet based on the Zhang's calibration method to calibrate the camera 130, and obtains the camera internal parameter matrix and the distortion coefficients (k1, k2, p1, p2, k3), as shown in Table 1.

[0122] Table 1 Camera 130 parameters

[0123] Camera internal parameter matrix

[0124] Camera 130 distortion D = (0.04902, -0.19261, 0.00029, -0.00045, 0)

[0125] Coefficient

[0126] Based on the internal camera parameter matrix K and distortion coefficient D calibrated in the above steps, the distorted correction of the acquired image is completed to obtain an undistorted image.

[0127] In the implementation process of the present invention, ArUco code is selected as the feature marker, and squares are added at the four corner positions of the ArUco code to enhance the robustness of the extraction of the outer corner points of the marker. The size is selected as 45*45mm, and id = 1. Determine the vertical distance h between the center of the tube orifice circle of the test tube 200 and the upper surface of the test tube rack 110 t = 70mm, and the distance x between the centers of adjacent tube orifices d = 30mm, to obtain the relative pose parameters of the plane O of the tube orifice circle of the test tube 200 b relative to the O a coordinate system Among them can be set as the identity matrix.

[0128] After determining all the basic parameters, in order to reflect the universality of the present invention, the camera 130 is placed obliquely to ensure that the image area can completely present the overall view of the test tube rack 110 and the test tube 200, and the test tube 200 is randomly placed in the test tube rack 110. The pose of the square reference marker 120 is extracted through the PnP algorithm to obtain the rotation matrix and translation vector as Figure 7 shown.

[0129] After obtaining the parameters and substituting them into the pose transformation formula it can be obtained The homography matrix H is solved by the method mentioned above, and finally the image plane is linearly mapped to obtain the plane of the tube orifice circle contour 150, as Figure 8 shown.

[0130] According to the known homography matrix H and the image pixel coordinates P of the center of the tube orifice circle of the test tube 200 i it can be obtained and solved P i ′ as a two-dimensional coordinate and indexing function.

[0131] After obtaining the plane image of the tube orifice circle contour 150, contour extraction is performed on the image to obtain a circular contour as Figure 9 shown. When the P i ′ point meets the Euclidean distance judgment condition, it is displayed in a solid circle in Figure 6 .

[0132] The index of the target test tube 200 has been obtained in the above steps. According to the index, the position and orientation of the center of the circular mouth of the test tube 200 under the benchmark of the test tube rack 110 can be traced as Furthermore, according to the position and orientation transformation, the position and orientation in the coordinate system of the camera 130 can be obtained as

[0133] It can be known from the above steps the position and orientation of the test tube rack 110 relative to the camera 130 It can be known from the above steps the position and orientation of the test tube 200 relative to the camera 130 By modeling and constraining reasonable and accurate target reference points, it is possible to Figure 10 demonstrate the relationship among the three in the case of three-dimensional simulation.

[0134] Please refer to Figure 11 , the second object of the present invention is to provide a positioning device for the test tube 200 based on an auxiliary marker. The device includes a test tube rack 110, a square reference marker 120, and a camera 130. The test tube rack 110 is formed with a plurality of placement slots 111 for placing the test tubes 200; the square reference marker 120 is arranged beside the placement slots 111; the camera 130 is used to capture the relevant image information of the square reference marker 120 on the test tube rack 110 and the test tubes 200 placed on the test tube rack 110.

[0135] In a possible implementation manner, the square reference marker 120 includes an ArUco marker, an AprilTag marker, or an ARTag marker. In the above steps, since there are many types of square reference markers 120, in the present invention, only the relatively typical ArUco marker is adopted. Similarly, an AprilTag two-dimensional code marker can also be introduced as a feature in the present invention to replace the ArUco marker for pose extraction.

[0136] The above-described embodiments only represent several implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation to the scope of the patent application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several deformations and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention patent shall be subject to the appended claims.

Claims

1. A test tube positioning method based on auxiliary marking, characterized in that, The method includes: S1. Calibrate the internal parameters of the camera, correct the imaging distortion using the calibrated internal parameters, and clarify the parameter dimensions of the test tube and the test tube rack; S2. Perform id recognition on the square reference mark and extract the feature corner points, and solve the pose of the mark through the PnP algorithm to determine the relative reference pose of the test tube rack; S3. Solve the plane of the test tube orifice circular contour by mapping with the homography matrix, solve the ideal center coordinate based on the contour circle extraction algorithm, and use it as the test tube index parameter; S4. Identify the orifice circular contour of the target test tube, calculate the Euclidean distance between the detected center coordinate and the test tube index parameter, and determine whether the test tube exists according to this distance; S5. Calculate the spatial pose of the test tube in the camera coordinate system, and then perform a three-dimensional simulation rendering of the overall physical system.

2. The test tube positioning method based on auxiliary marking according to claim 1, wherein, The step S1 includes: S11. Use the camera to capture multiple plane target images at different angles, and perform camera calibration using the Zhang's calibration method to obtain the camera internal parameter matrix and the distortion coefficient; S12. Complete the distortion correction of the acquired images based on the calibrated camera internal parameter matrix and the distortion coefficient; S13. Determine the ID and overall dimensions of the square reference mark, fix the square reference mark in the blank area on the upper surface of the test tube rack, use the upper left corner coordinates of the square reference mark as the reference point of the test tube rack, set up the camera coordinate system, select the specification size of the test tube, and at the same time set the vertical distance between the center of the tube mouth circle of the test tube and the upper surface of the test tube rack as h t , and the distance between the centers of the tube mouths of adjacent test tubes is x d .

3. The test tube positioning method based on auxiliary marking according to claim 2, wherein, The step S2 includes: S21. Obtain the id of the mark and the pixel coordinates of the four outer corner points through the mark recognition algorithm; S22. Combine the known mark size, use the PnP algorithm to solve the rotation matrix and translation vector of the mark relative to the camera coordinate system, and use this pose as the pose of the test tube rack relative to the camera reference.

4. The test tube positioning method based on auxiliary marking according to claim 1, wherein The step S3 includes: S31. Calculate the rotation matrix and translation vector of the test tube orifice circular contour plane relative to the camera coordinate system according to the pose relationship between the reference coordinate system of the test tube orifice circular contour plane and the camera coordinate system; S32. Use the homography matrix to map the image plane to the test tube orifice circular contour plane; S33. Solve the ideal center coordinate based on the contour circle extraction algorithm, and use the ideal center coordinate as the test tube index parameter.

5. The test tube positioning method based on auxiliary marking according to claim 1, wherein, The step S4 includes: S41. Perform circular contour detection on the image of the test tube orifice circular contour plane to obtain all possible circular contours in the image; S42. Filter out abnormal contours according to the threshold constraint to obtain the remaining test tube orifice circular contours; S43. Calculate the Euclidean distance between the detected center coordinate and the test tube index parameter; S44. When this distance meets the set threshold condition, judge that there is a test tube at this position; S45. Obtain the storage situation of the test tube in this scenario by tracing the index sequence of this circular contour on the test tube rack.

6. The test tube positioning method based on auxiliary marking according to claim 5, wherein The formula for the Euclidean distance judgment condition is: Among them, is the center coordinate of the detected circular contour of the test tube mouth, P i ′(u i ′, v i ′) is the pixel coordinate of the center of the circular test tube mouth in the ideal state, r j is the radius of the contour circle.

7. The test tube positioning method based on auxiliary marking according to claim 1, characterized in that The calculation formula for the homography matrix H is: H = K[r1 r2 t] where K is the camera internal parameter matrix, r1 and r2 are the first two columns of the rotation matrix, and t is the translation vector.

8. A test tube positioning device based on auxiliary markings, characterized in that, The device includes: A test tube rack formed with a plurality of placement slots for placing test tubes; A square reference mark disposed beside the placement slot of the test tube rack; A camera for capturing relevant image information of the square reference mark on the test tube rack and the test tubes placed on the test tube rack.

9. The test tube positioning device based on auxiliary marking according to claim 8, wherein, The types of the square reference marks include ArUco marks, AprilTag marks or ARTag marks.

Images