High-precision spotting system and method based on machine vision and laser displacement guidance

Abstract

The present invention discloses a high-precision spotting system and method based on machine vision and laser displacement guidance, wherein the system includes a vision system and a laser displacement sensor. The present invention uses machine vision to accurately locate the detection chip, uses a machine vision algorithm to accurately guide the spotting needle, and places the sample at the target spotting point. At the same time, the height of the detection chip relative to the spotting needle is measured by a laser displacement sensor, and the spotting height of each target spotting point is corrected, thereby avoiding solution residue at the needle tip and the spotting needle tip scratching the surface of the chip to be detected.

Description

Technical Field

[0001] The present invention relates to the field of biochip technology, and more specifically, to a high-precision spotting system and method based on machine vision and laser displacement guidance. Background Art

[0002] In the process of drug screening, it is necessary to know which specific protein the drug reacts with, and there are many types of these proteins. The usual practice is to spot various proteins on a detection chip through a spotting device, which presents a dense array of various proteins on the detection chip.

[0003] The currently commonly used spotting device mainly places the detection chip at the chip spotting position by positioning it against the side, manually controls the XYZ axis, moves the spotting needle, aligns it with the starting position of the detection chip spotting, and records the starting position on the control software. Then, the sampling needle is driven by the XYZ platform to take samples from the corresponding centrifuge tube to achieve automatic spotting. However, this positioning method is limited by the accuracy of the chip housing. Among them, the processing error of the detection chip housing, the relative position deviation between the detection chip housing and the detection chip, and the position error of the manually selected starting point will affect the accuracy of the spotting. In addition, subsequent detection still needs to adjust the detection position, which makes the detection process complicated, thereby affecting the accuracy of the detection. Summary of the invention

[0004] In view of this, the present invention provides a high-precision spotting system and method based on machine vision and laser displacement guidance, which are used to solve the problem of poor positioning accuracy of existing spotting devices and methods.

[0005] To achieve the above objectives, the proposed solution is as follows: A high-precision spotting system based on machine vision and laser displacement guidance, the system comprising: a vision system, a laser displacement sensor and a control unit; The visual system is used to take pictures and locate the chip mark position; The control unit is used to calculate the target spot position according to the detection chip mark position; The laser displacement sensor is used to measure the target spot height of each target spot; The control unit is used to control the XYZ motion platform to drive the spotting needle to spot based on the target spotting position and spotting height.

[0006] A high-precision spotting method based on machine vision and laser displacement guidance is applied to the above-mentioned spotting system, and the method includes: Place the test chip at the spotting position, and the visual system takes a picture of the test chip and locates the mark position of the test chip; The control unit calculates the target spotting position according to the input theoretical spotting position of the theoretical chip, the theoretical marking position and the marking position of the detection chip; The control unit controls the XY axis of the XYZ motion platform to carry the laser displacement sensor to measure the target spot height of each target spot above the target spot of the detection chip; The control unit controls the XY axis to move to above the target spotting position of the target spotting point, and controls the Z axis to descend to the target spotting height corresponding to the target spotting point for spotting.

[0007] Preferably, before the visual system takes a picture of the detection chip, the method further includes: The vision system calibrates the pixel coordinate system to the mechanical coordinate system of the spotting device.

[0008] Preferably, the process of calibrating the pixel coordinate system to the mechanical coordinate system of the spotting device comprises: Install the pen at the spotting needle position and stick white paper on the spotting platform; The control unit controls the XYZ motion platform to drive the pen to draw 9 circles on the white paper and record the mechanical coordinates of the center of each circle; The visual system takes a photo of the white paper and calculates the pixel coordinates corresponding to the center of each circle; The hand-eye calibration matrix is ​​calculated based on the pixel coordinates and mechanical coordinates corresponding to each circle center.

[0009] Preferably, the process of calculating the target spotting position according to the input theoretical spotting position of the theoretical chip, the theoretical marking position and the marking position of the detection chip comprises: An affine transformation matrix is ​​calculated based on the theoretical marking position and the detection chip marking position; The target sampling position is obtained by calculating according to the affine transformation matrix and the theoretical sampling position.

[0010] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: The invention provides a high-precision spotting system and method based on machine vision and laser displacement guidance, the system includes a vision system and a laser displacement sensor. The invention uses machine vision to locate the detection chip, uses a machine vision algorithm to accurately guide the spotting needle, and places the sample at the target spotting point.

[0011] The laser displacement sensor of the present invention measures the height of the detection chip relative to the spotting needle, and corrects the spotting height of each target spotting point, thereby preventing the solution from remaining on the needle tip and the spotting needle tip from scratching the surface of the chip to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0013] Figure 1 A schematic diagram of the structure of a high-precision spotting system based on machine vision and laser displacement guidance provided by an embodiment of the present invention; Figure 2 A schematic diagram of a height correction process of a laser displacement sensor provided by an embodiment of the present invention; Figure 3 A schematic diagram of a hand-eye calibration process provided by an embodiment of the present invention; Among them, 1 is a visual system, 2 is a laser displacement sensor, 3 is a spotting needle, 4 is a centrifuge tube rack, 5 is a centrifuge tube, 6 is a detection chip, 7 is a sampling needle rack, and 8 is a syringe pump. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] First, combine Figure 1 The high-precision spotting system based on machine vision and laser displacement guidance provided by the embodiment of the present invention is introduced. Figure 1 As shown, the spotting system of the embodiment of the present invention includes: a visual system 1, a laser displacement sensor 2 and a control unit on the basis of the spotting needle 3, centrifuge tube rack 4, centrifuge tube 5, detection chip 6, sampling needle rack 7 and injection pump 8 of the traditional spotting system.

[0016] The visual system 1 is used to take pictures, and to capture the detection chip 6 through a machine vision algorithm, and to calculate the marking position of the detection chip 6 .

[0017] The control unit is used to calculate the target spot position according to the marked position of the detection chip 6. The control unit includes control software.

[0018] The laser displacement sensor 2 is used to measure the target spot height of each target spot.

[0019] The control unit is used to control the XYZ motion platform to drive the spotting needle 3 to spot samples based on the target spotting position and spotting height.

[0020] The invention provides a high-precision spotting system and method based on machine vision and laser displacement guidance, the system includes a vision system and a laser displacement sensor. The invention uses machine vision to locate the detection chip, uses a machine vision algorithm to accurately guide the spotting needle, and places the sample at the target spotting point.

[0021] The laser displacement sensor of the present invention measures the height of the detection chip relative to the spotting needle, and corrects the spotting height of each target spotting point, thereby preventing the solution from remaining on the needle tip and the spotting needle tip from scratching the surface of the chip to be detected.

[0022] Next, the embodiment of the present invention introduces a high-precision spotting method based on machine vision and laser displacement guidance. The spotting method is implemented based on the spotting system of the above embodiment, and the method includes: Step S1 , placing the detection chip 6 at the spotting position, the visual system 1 takes a picture of the detection chip 6 , and locates the marking position of the detection chip 6 .

[0023] Specifically, the detection chip 6 is placed at the spotting position, and the visual system 1 takes a picture of the detection chip 6 . The visual system 1 captures the mark of the detection chip 6 through a machine learning algorithm and locates the mark position of the detection chip 6 .

[0024] The visual system 1 takes a photo to locate the detection chip 6, and accurately finds the pixel coordinates MarkP (X, Y, Theta) corresponding to the current detection chip 6 mark Mark by performing template matching, Hough line finding, and Hough line finding algorithm based on the contour of the detection chip 6. The pixel coordinates are transformed through the preset hand-eye calibration matrix to obtain the coordinates NowMark[X, Y, Theta] of the pixel coordinates of the detection chip 6mark in the mechanical coordinate system. The transformation process is as follows: NowMark[X,Y,Theta] = MatUpCamToW * MarkP(X,Y,Theta) Among them, MatUpCamToW is the preset hand-eye calibration matrix.

[0025] Step S2, the control unit calculates the target spotting position according to the input theoretical spotting position of the theoretical chip, the theoretical marking position and the marking position of the detection chip 6.

[0026] Specifically, the theoretical spotting position and theoretical marking position of the theoretical chip are input into the control software of the spotting system control unit. Then, according to the marking position of the detection chip 6 located by the visual system 1 and the input theoretical marking position, the affine transformation matrix of the marking point position of the current detection chip 6 relative to the theoretical marking point position is calculated. Finally, the theoretical spotting position is translated by the affine transformation matrix to calculate the target spotting position. The target spotting position is obtained by calculation based on the affine transformation matrix and the theoretical spotting position.

[0027] The affine transformation matrix calculation process is as follows: VectorAngleMat=vector_angle_to_rigid(Mark0[X,Y,Theta],NowMark[X,Y,Theta]); Among them, VectorAngleMat is the affine transformation matrix, vector_angle_to_rigid is the rigid affine transformation function, and Mark0[X,Y,Theta] is the theoretical marker position.

[0028] The target point position calculation process is as follows: NowPts[X, Y] = VectorAngleMat*Pts0[X, Y]; Among them, Pts0[X, Y] is the theoretical sampling position, and NowPts[X, Y] is the target sampling position.

[0029] The present invention uses a machine vision algorithm to accurately guide the spotting needle to spot the sample to the corresponding target spotting point, ensuring that the spotting position error of each spotting point is less than 0.02mm.

[0030] Step S3 , the control unit controls the XY axis of the XYZ motion platform to carry the laser displacement sensor 2 to measure the target spot height of each target spot above the target spot of the detection chip 6 .

[0031] Specifically, Figure 2 As shown, the control unit controls the XY axis of the XYZ motion platform to carry the laser displacement sensor 2 above the target sampling point corresponding to the detection chip 6, and fine-tunes the height of the Z axis when each target sampling is performed to measure the target sampling height of each target sampling point. The adjusted Z axis height coordinates of each target sampling point are recorded to obtain the target sampling position NowPt[X,Y,Z] including the target sampling height.

[0032] Step S4, the control unit controls the XY axis to move to above the target spotting position of the target spotting point, and controls the Z axis to descend to the target spotting height corresponding to the target spotting point to perform spotting.

[0033] Specifically, the control unit controls the XYZ motion platform to install the spotting needle 3 on the sampling needle holder 7. The XYZ motion platform carries the spotting needle 3 to the centrifuge tube 5 on the centrifuge tube holder 4 to take samples. The control unit controls the XY axis of the XYZ motion platform to drive the spotting needle 3 to move above a target spotting point on the detection chip 6, and then the control unit controls the Z axis to drive the spotting needle 3 down to the target spotting height corrected by the laser displacement sensor 2, so that the position of the needle tip of the spotting needle 3 can be controlled to be within 0.05±0.02 mm from the position of the surface of the detection chip 6. Finally, the control unit controls the injection pump 8 to extrude the sample and spot it on the surface of the detection chip 6 to complete the spotting of the target spotting point. The XYZ platform drives the sampler to move to the needle throwing position and throws away the used spotting needle 3. Repeat step 4 until the spotting of all target spotting points is completed.

[0034] The present invention uses a laser displacement sensor to correct the spotting height of each spotting point, ensuring that the height error of each spotting point is controlled within 0.02mm, thereby preventing the solution from remaining on the needle tip and the spotting needle tip from scratching the chip surface.

[0035] Considering that the embodiment of the present invention needs to convert the pixel coordinate system and the mechanical coordinate system, the embodiment of the present invention may further include the following steps before the visual system 1 of the aforementioned embodiment takes a picture of the detection chip 6: The vision system 1 calibrates the pixel coordinate system to the mechanical coordinate system of the spotting device.

[0036] Specifically, the pixel coordinate system is calibrated to the mechanical coordinate system of the spotting system through a nine-point calibration method commonly used in machine vision.

[0037] Remove the spotting needle, install the special marker pen at the spotting needle position, and stick white paper on the spotting platform. The control unit controls the XYZ motion platform to drive the pen to draw on the white paper. Figure 3 The nine circles shown in the figure are marked with the mechanical coordinates of the center of each circle. The visual system 1 takes a photo of the white paper for positioning, and calculates the pixel coordinates corresponding to the center of each circle using the Hough circle finding algorithm. The hand-eye calibration matrix is ​​calculated based on the nine-point calibration algorithm, the pixel coordinates corresponding to the center of each circle, and the mechanical coordinates. The calculation process of the hand-eye calibration matrix is ​​as follows: MatUpCamToW = vector_to_hom_mat2d(MarkP(P x ,P y ), MarkW(W x , W y )) Among them, MarkW(W x , W y ) is the mechanical coordinate corresponding to the center of the circle, RotCenterP(P x ,P y) is the pixel coordinate corresponding to the center of the circle, MatUpCamToW is the hand-eye calibration matrix, and vector_to_hom_mat2d is the coordinate conversion function commonly used in the nine-point calibration method.

[0038] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0039] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0040] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A high-precision spotting system based on machine vision and laser displacement guidance, characterized in that: The system includes: a visual system, a laser displacement sensor and a control unit; The visual system is used to take pictures and locate the chip mark position; The control unit is used to calculate the target spot position according to the detection chip mark position; The laser displacement sensor is used to measure the target spot height of each target spot; The control unit is used to control the XYZ motion platform to drive the spotting needle to spot based on the target spotting position and spotting height.

2. A high-precision spotting method based on machine vision and laser displacement guidance, characterized in that: Applied to the spotting system of claim 1, the method comprising: Place the test chip at the spotting position, and the visual system takes a picture of the test chip and locates the mark position of the test chip; The control unit calculates the target spotting position according to the input theoretical spotting position of the theoretical chip, the theoretical marking position and the marking position of the detection chip; The control unit controls the XY axis of the XYZ motion platform to carry the laser displacement sensor to measure the target spot height of each target spot above the target spot of the detection chip; The control unit controls the XY axis to move to above the target spotting position of the target spotting point, and controls the Z axis to descend to the target spotting height corresponding to the target spotting point for spotting.

3. The high-precision spotting method based on machine vision and laser displacement guidance according to claim 2 is characterized in that: Before the visual system takes a picture of the detection chip, it also includes: The vision system calibrates the pixel coordinate system to the mechanical coordinate system of the spotting device.

4. The high-precision spotting method based on machine vision and laser displacement guidance according to claim 3 is characterized in that: The process of calibrating the pixel coordinate system to the mechanical coordinate system of the spotting device includes: Install the pen at the spotting needle position and stick white paper on the spotting platform; The control unit controls the XYZ motion platform to drive the pen to draw 9 circles on the white paper and record the mechanical coordinates of the center of each circle; The visual system takes a photo of the white paper and calculates the pixel coordinates corresponding to the center of each circle; The hand-eye calibration matrix is ​​calculated based on the pixel coordinates and mechanical coordinates corresponding to each circle center.

5. The high-precision spotting method based on machine vision and laser displacement guidance according to claim 2, characterized in that: The process of calculating the target spotting position according to the input theoretical spotting position of the theoretical chip, the theoretical marking position and the marking position of the detection chip includes: An affine transformation matrix is ​​calculated based on the theoretical marking position and the detection chip marking position; The target sampling position is obtained by calculating according to the affine transformation matrix and the theoretical sampling position.

Images