A method and apparatus for detecting the volume of trace liquids based on machine vision

By using a machine vision-based approach and employing light processing and image processing algorithms to identify the surface of trace liquids, the problems of low detection accuracy and cross-contamination in existing technologies are solved, achieving high-precision, non-contact detection of trace liquid volume and liquid replenishment path planning.

CN123111403APending Publication Date: 2026-11-17YONGYUAN HOPSON TECH (SHENZHEN) CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202611561889.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-10
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing technologies for detecting trace liquids suffer from problems such as cross-contamination risk, slow detection speed, structural wear, and severe perspective distortion, making it impossible to achieve high-precision, non-contact, real-time online detection.

Method used

A machine vision-based approach is adopted to acquire images of the target plate, eliminate reflection interference from the inclined hole walls using light processing components, and combine image processing algorithms such as Gaussian filtering, histogram equalization, and distortion correction to identify the liquid surface and calculate the area. The volume is converted using calibration relationships to achieve non-contact detection, and a liquid replenishment path is planned when a deviation is detected.

Benefits of technology

It achieves high-precision, non-contact micro-liquid volume detection, eliminates the risk of cross-contamination, improves detection speed and accuracy, and ensures product quality control.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123111403A_ABST
    Figure CN123111403A_ABST
Patent Text Reader

Abstract

This invention discloses a method and apparatus for detecting the volume of trace liquids based on machine vision. The method includes: acquiring an image of a target plate containing liquid, wherein the target plate has several holes; identifying the liquid surface in the holes by image recognition and calculating the actual area of ​​the liquid surface; converting the area of ​​the actual liquid surface into the actual volume based on a preset calibration relationship, comparing the actual volume with a preset target volume to obtain the volume deviation, and sending the volume deviation as communication data to a control system. This invention uses machine vision for non-contact detection of the liquid volume within the holes and sends the volume deviation to the control system, facilitating automated detection and closed-loop control.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of biosynthesis detection technology, specifically relating to a method and device for detecting the volume of trace liquids based on machine vision. Background Technology

[0002] With the rapid development of high-throughput biosynthesis (such as solid-phase synthesis of long-fragment DNA), high-throughput drug screening, and ultra-micro chemical analysis technologies, extremely high demands are placed on the precise dispensing of trace liquids and the rapid online detection of dispensing volumes. During solid-phase synthesis or dispensing processes, the large number of jet channels and the tiny volume of each dispensing operation make it highly susceptible to abnormalities such as insufficient dispensing, leakage, liquid residue buildup, or blockage. Therefore, achieving high-precision, non-contact, real-time online detection of the volume of trace liquids within the wells of multi-well plates has become crucial for ensuring synthesis yield and product quality control.

[0003] Currently, the following two technical solutions are mainly used for the volume detection of trace liquids within porous plates, both domestically and internationally:

[0004] For example, the technical solution in patent application number CN201510123456.7, entitled "A Trace Liquid Detection Device," utilizes capacitive or pressure-based liquid level sensing technologies for traditional trace liquid volume detection. These technologies involve a probe contacting the liquid, using capacitance changes or pressure feedback to determine the liquid level position and then calculating the liquid volume. For instance, the detection probe descends and contacts the liquid surface, generating an electrical or pressure signal, which is then used to calculate the volume based on the liquid level height and the orifice plate geometry.

[0005] As described in the journal *Optics and Precision Engineering*, Volume 28, Issue 5, pp. 1123-1130, 2020: High-end equipment uses ordinary cameras to capture images of the liquid surface and analyzes its features using image processing algorithms. Using ordinary industrial lenses and a ring light source, edge extraction and area calculation are performed after acquiring the liquid surface image. Specifically, the process involves capturing images with an ordinary camera, then illuminating the surface with a ring light source, extracting the liquid surface contour using an edge detection algorithm, and finally estimating the volume based on the pixel area.

[0006] However, the aforementioned existing technologies have the following technical problems: in traditional contact liquid surface detection technologies, there are risks of cross-contamination caused by direct contact between the probe and the reagent; slow detection speed caused by the mechanical probe moving up and down one hole at a time; structural wear caused by frequent mechanical lifting and lowering; and physical disturbance of the probe to the micro-level liquid surface. In addition, in the use of ordinary lenses in ordinary optical imaging detection technologies, there are serious perspective distortions and insufficient depth of field that prevents the liquid surface at different depths of the multi-hole plate from being clearly imaged simultaneously. Summary of the Invention

[0007] To address the aforementioned technical problems in the prior art, this invention provides a method for detecting the volume of trace liquids based on machine vision, the method comprising the following steps:

[0008] Acquire an image of a target plate containing liquid, wherein the target plate has a plurality of holes;

[0009] The image is identified to segment the liquid surface in the orifice and the area of ​​the actual liquid surface is calculated.

[0010] Based on a preset calibration relationship, the area of ​​the actual liquid surface is converted into the actual volume, and the actual volume is compared with the preset target volume to obtain the volume deviation. The volume deviation is then sent to the control system as communication data.

[0011] Specifically, the step of acquiring an image of the target plate containing the liquid includes:

[0012] The target plate is illuminated using a light processing component, causing the light illuminating the inclined hole wall at the hole location to be scattered.

[0013] The light illuminating the liquid surface is reflected vertically back to the camera to eliminate the glare interference from the inclined orifice wall, and the image is acquired through the camera.

[0014] Specifically, the step of acquiring an image of the target plate containing the liquid further includes:

[0015] The control system drives the target board to move to the testing station.

[0016] When the target board moves to the inspection station, the camera is triggered to acquire an image, and at least one of the camera's exposure time, gain, and white balance parameters is adjusted to obtain an original full-board image containing the hole positions.

[0017] Specifically, the step of recognizing the image to segment the liquid surface in the orifice and calculating the area of ​​the actual liquid surface includes:

[0018] The acquired image is preprocessed, and the preprocessing method is at least one of Gaussian filtering for noise reduction, histogram equalization, or distortion correction.

[0019] An algorithm is used to identify at least one of the center coordinates or radii of each of the aforementioned hole positions;

[0020] Each of the holes after positioning is defined as the search area;

[0021] Extract the initial annular edge where the meniscus intersects the borehole wall;

[0022] Centered on the central pixel inside the initial annular edge, the process extends outward to the initial annular edge to segment out a circular region of the liquid surface.

[0023] The algorithm is at least one of template matching, circle detection, boundary detection, geometric registration, or training model.

[0024] Specifically, the steps for calculating the actual liquid surface area include:

[0025] Calculate the number of pixels contained in the segmented liquid surface;

[0026] Using the calibrated conversion factor, the area of ​​the actual liquid surface is calculated according to the following formula:

[0027] S = N × C;

[0028] Where S is the area of ​​the actual liquid surface, in mm. 2 N represents the number of pixels; C represents the pixel physical size conversion factor, in mm. 2 / pixel.

[0029] Specifically, the step of converting the area of ​​the actual liquid surface into the actual volume based on a preset calibration relationship includes:

[0030] The actual liquid surface area is substituted into a preset volume and area fitting curve for calculation to obtain the actual volume.

[0031] Specifically, the step of comparing the actual volume with the preset target volume to obtain the volume deviation, and sending the volume deviation as communication data to the control system includes:

[0032] The volume deviation rate, which is the volume deviation, is calculated.

[0033] The communication data is encapsulated as at least one of the following parameters: borehole coordinates, actual volume, preset target volume, and volume deviation rate, and then sent to the control system.

[0034] In some embodiments, the machine vision-based method for detecting the volume of trace liquids further includes the following steps:

[0035] The control system plans a replenishment path based on the volume deviation; the step of the control system planning the replenishment path based on the volume deviation includes:

[0036] Set a qualified threshold for the volume deviation rate, and determine whether the calculated volume deviation rate exceeds the qualified threshold.

[0037] If the volume deviation rate exceeds the acceptable threshold, the replenishment volume is calculated as follows:

[0038] The replenishment path is planned based on the replenishment volume, the physical coordinates of the target plate, and the magnetic levitation mover supporting the target plate.

[0039] The target plate is replenished with liquid according to the described replenishment path.

[0040] Specifically, the step of performing the liquid replenishment operation on the target plate according to the liquid replenishment path includes:

[0041] Based on the replenishment path, determine the pipetting valve island number, pipetting channel number, and the identification mark of the magnetically levitated actuator carrying the target plate;

[0042] The magnetic levitation mover is controlled to move along the magnetic levitation track to transport the hole in the target plate that needs to be replenished to the direct below the corresponding pipetting channel;

[0043] The pipetting valve island is controlled to translate and align, and the corresponding pipetting channel is driven to perform injection to replenish the liquid volume to the corresponding orifice.

[0044] This application also provides a machine vision-based device for detecting the volume of trace liquids, comprising:

[0045] An image acquisition module is configured to acquire an image of a target plate containing liquid, wherein the target plate has a plurality of holes;

[0046] The processing module is configured to recognize the image to segment the liquid surface in the hole and calculate the area of ​​the actual liquid surface;

[0047] The judgment module is configured to convert the area of ​​the actual liquid surface into the actual volume based on a preset calibration relationship, compare the actual volume with the preset target volume to obtain the volume deviation, and send the volume deviation as communication data to the control system.

[0048] In some embodiments, the machine vision-based micro-liquid volume detection device further includes:

[0049] The execution module is configured to, when the volume deviation exceeds a set range, have the control system plan a replenishment path based on the volume deviation and perform a replenishment operation on the target plate according to the replenishment path. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0051] Figure 1 This is a flowchart of a machine vision-based method for detecting the volume of trace liquids provided in this application;

[0052] Figure 2 What is provided is as follows Figure 1 The flowchart of step S1 is shown below;

[0053] Figure 3 yes Figure 1 A flowchart illustrating the specific steps of step S1 in another embodiment of the present invention;

[0054] Figure 4 yes Figure 1 A flowchart illustrating the specific steps of step S2 in the diagram;

[0055] Figure 5 yes Figure 1 A flowchart illustrating step S2 in another embodiment of the present invention;

[0056] Figure 6 yes Figure 1 A flowchart illustrating the specific steps of step S3 in the process;

[0057] Figure 7 yes Figure 1 A flowchart illustrating the specific steps of step S3 in the process;

[0058] Figure 8 This is a schematic diagram of a fluid rehydration pathway planning process provided in this application;

[0059] Figure 9 This is a flowchart of another machine vision-based method for detecting the volume of trace liquids provided in this application;

[0060] Figure 10 Is it like this? Figure 9 The flowchart of step S04 is shown below;

[0061] Figure 11 This application provides a machine vision-based device for detecting the volume of trace liquids.

[0062] Figure 12 This application provides another machine vision-based micro-liquid volume detection device.

[0063] Figure 13This is a schematic diagram of the image acquisition module 01;

[0064] Figure 14 Is it like this? Figure 13 The diagram shows the structure of the processing module 02.

[0065] Figure 15 Is it like this? Figure 13 A schematic diagram of the processing module 02 of another embodiment is shown;

[0066] Figure 16 Is it like this? Figure 13 The diagram shown is a structural schematic of the judgment module 03.

[0067] Figure 17 This is a schematic diagram of the structure of the execution module 04 provided in this application;

[0068] Figure 18 This application provides a machine vision-based system for detecting the volume of trace liquids. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0070] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0071] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0072] In this invention, unless otherwise explicitly specified and limited, the terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0073] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0074] like Figure 1 As shown, Figure 1 A flowchart is provided for a machine vision-based method for detecting the volume of trace liquids, the method comprising the following steps:

[0075] S1. Acquire an image of a target plate containing liquid, wherein the target plate has a plurality of holes;

[0076] S2. Recognize the image to segment the liquid surface in the hole and calculate the area of ​​the actual liquid surface;

[0077] S3. Based on the preset calibration relationship, the area of ​​the actual liquid surface is converted into the actual volume, and the actual volume is compared with the preset target volume to obtain the volume deviation. The volume deviation is then sent to the control system as communication data.

[0078] The communication data can include volume deviation, actual volume, target volume, relative deviation, deviation level, pass / fail status, orifice number, target plate ID, liquid surface area, calibration coefficient, timestamp, alarm information, control request, suggested adjustment amount, confidence level, abnormal flag, etc. The communication form can be wired or wireless, specifically USB, Ethernet, Wi-Fi, Bluetooth, etc.

[0079] The control system can be a PLC, PAC, industrial computer, embedded controller, motion controller, pipetting workstation controller, liquid dispenser controller, robotic arm controller, sorting and rejection controller, alarm controller, SCADA system, MES system, LIMS system, host computer, server, or cloud control platform. Its control actions can be continue, pause, alarm, mark, reject, replenish, reduce, adjust the liquid volume, re-inspect, record data, or upload to the cloud, etc.

[0080] like Figure 2 As shown, Figure 2 What is provided is as follows Figure 1 The flowchart of step S1 shown includes the following steps:

[0081] S11. Irradiate the target plate using a light processing component to scatter the light irradiated on the inclined hole wall of the hole position;

[0082] S12. The light illuminating the liquid surface is reflected vertically back to the camera to eliminate the reflection interference from the inclined hole wall, and the image is acquired through the camera.

[0083] It should be specifically noted that, depending on the illumination method and lens type, the light processing components can be mainly divided into the following categories: telecentric lenses and LED coaxial light sources, low-angle ring lights and fixed-focus lenses, high-angle ring lights and fixed-focus lenses, backlighting and telecentric / fixed-focus lenses, dome lights and fixed-focus lenses, and implementation methods such as bar lights / line array lights and line array cameras.

[0084] like Figure 3 As shown, Figure 3 yes Figure 1 A flowchart illustrating the specific steps of step S1 in another embodiment of the invention is shown, which includes:

[0085] S011. The target board is moved to the testing station by the control system.

[0086] S012. When the target board moves to the inspection station, the camera is triggered to acquire an image, and at least one of the camera's exposure time, gain, and white balance parameters is adjusted to obtain an original full-board image containing the hole positions.

[0087] like Figure 4 As shown, Figure 4 yes Figure 1 A flowchart illustrating the specific steps of step S2 is provided, which includes:

[0088] S21. The acquired image is preprocessed, and the preprocessing method is at least one of Gaussian filtering for noise reduction, histogram equalization, and distortion correction.

[0089] S22. The center coordinates and radius of each hole are identified using an algorithm;

[0090] S23. Set each of the holes after positioning as the search area;

[0091] S24. Extract the initial annular edge where the meniscus intersects with the borehole wall;

[0092] S25. Taking the center pixel inside the initial annular edge as the center, extend outward to the initial annular edge to divide the circular area of ​​the liquid surface.

[0093] The algorithm is at least one of template matching, circle detection, boundary detection, geometric registration, or training model.

[0094] The template matching algorithm involves sliding a predefined standard template image onto the image to be tested. By calculating the similarity between the template and the covered area (such as normalized cross-correlation), the position with the highest similarity is found, which is the target position. Essentially, it's a correlation search based on pixel grayscale.

[0095] Among them, the circle detection algorithm usually uses Hough circle transform to map the edge pixels in the image to the three-dimensional parameter space of the center-radius and vote. If the number of votes exceeds the threshold, it is determined to be a circle, and the center coordinates and radius are output.

[0096] Among them, the boundary detection algorithm locates the position of pixels where the gray level changes drastically by calculating the gradient magnitude and direction of the image gray level, and outputs a binary edge image (such as the Canny algorithm).

[0097] Among them, the geometric registration algorithm finds the optimal spatial transformation matrix (translation, rotation, scaling, etc.) to map the feature points in the image to be tested to the corresponding points in the reference image (or CAD model), thereby achieving alignment between the two.

[0098] The training model algorithm utilizes a large amount of labeled hole location image data to train a deep neural network (such as YOLO or CNN), enabling the network to automatically learn the mapping relationship from image pixels to target locations. During inference, the input image is the network, which directly outputs the hole location coordinates or a segmentation mask.

[0099] like Figure 5 As shown, Figure 5 yes Figure 1 A flowchart of step S2 in another embodiment may further include:

[0100] S21. The acquired image is preprocessed, and the preprocessing method is at least one of Gaussian filtering for noise reduction, histogram equalization, and distortion correction.

[0101] S22. The center coordinates and radius of each hole are identified using an algorithm;

[0102] S23. Set each of the holes after positioning as the search area;

[0103] S24. Extract the initial annular edge where the meniscus intersects with the borehole wall;

[0104] S25. Taking the center pixel inside the initial annular edge as the center, extend outward to the initial annular edge to divide the circular area of ​​the liquid surface;

[0105] S26. Calculate the number of pixels contained in the segmented circular region of the liquid surface;

[0106] S27. Using the calibrated conversion coefficient, obtain the area of ​​the actual liquid surface;

[0107] In this embodiment, the area of ​​the actual liquid surface is calculated according to the following formula:

[0108] S = N × C;

[0109] Where S is the area of ​​the actual liquid surface, in mm. 2 N represents the number of pixels; C represents the pixel physical size conversion factor, in mm. 2 / pixel.

[0110] Specifically, in some embodiments, the step of converting the area of ​​the actual liquid surface into the actual volume based on a preset calibration relationship includes: substituting the area of ​​the actual liquid surface into a preset volume and area fitting curve for calculation to obtain the actual volume.

[0111] like Figure 6 As shown, Figure 6 yes Figure 1 A flowchart illustrating the specific steps of step S3 in the diagram. Step S3 includes:

[0112] S031. Calculate the volume deviation rate as the volume deviation;

[0113] S032. Encapsulate at least one of the following parameters as communication data: the coordinates of the hole position, the actual volume, the preset target volume, and the volume deviation rate, and send the communication data to the control system.

[0114] like Figure 7 As shown, Figure 7 yes Figure 1 A flowchart illustrating the specific steps of step S3. In some embodiments, step S3 includes:

[0115] S31. Substitute the area of ​​the actual liquid surface into a preset volume-area fitting curve to calculate the actual volume. The fitting curve satisfies the following formula:

[0116] V=a*S 3 +b*S 2 +c*S+d;

[0117] Where V is the actual volume, in μL; and S is the area of ​​the actual liquid surface, in mm. 2 a, b, c, and d are the fitting coefficients.

[0118] S32. Calculate the volume deviation rate, which is the volume deviation. The calculation formula is as follows:

[0119] δ=(V−V target ) / V target ×100%

[0120] Wherein, δ is the volume deviation rate; V is the actual volume; V target The preset target volume;

[0121] S33. Encapsulate at least one of the following parameters as communication data: the coordinates of the hole, the actual volume, the preset target volume, and the volume deviation rate, and send the communication data to the control system.

[0122] like Figure 8 As shown, Figure 8 This application provides a schematic diagram of a fluid resuscitation pathway planning process, including:

[0123] S41. Set a qualified threshold for the deviation rate of the volume deviation, and determine whether the calculated volume deviation rate exceeds the qualified threshold.

[0124] S42. If the volume deviation rate exceeds the acceptable threshold, the replenishment volume is calculated; the calculation formula is as follows:

[0125] V make up =V target -V;

[0126] Among them, V make up The volume of fluid replenishment is in μL; V target V represents the preset target volume in μL; V represents the actual volume in μL.

[0127] S43. Based on the replenishment volume, the physical coordinates of the target plate, and the magnetic levitation actuator carrying the target plate, plan the replenishment path.

[0128] like Figure 9 As shown, Figure 9 This is a flowchart of another machine vision-based method for detecting the volume of trace liquids provided in this application. The method includes the following steps:

[0129] S01. Acquire an image of a target plate containing liquid, wherein the target plate has a plurality of holes;

[0130] S02. Recognize the image to segment the liquid surface in the hole and calculate the area of ​​the actual liquid surface;

[0131] S03. Based on the preset calibration relationship, the area of ​​the actual liquid surface is converted into the actual volume, and the actual volume is compared with the preset target volume to obtain the volume deviation. The volume deviation is then sent to the control system as communication data.

[0132] S04. The control system plans the liquid replenishment path based on the volume deviation.

[0133] like Figure 10 As shown, Figure 10 Is it like this? Figure 9 The flowchart shown is for step S04. The method includes the following steps:

[0134] S041. Based on the replenishment path, determine the pipetting valve island number, pipetting channel number, and the identification mark of the magnetic levitation actuator carrying the target plate that performs the replenishment.

[0135] S042. Control the magnetic levitation mover to move along the magnetic levitation track to transport the hole in the target plate that needs to be replenished to the direct below the corresponding liquid transfer channel;

[0136] S043. Control the pipetting valve island to move and align, and drive the corresponding pipetting channel to perform injection, so as to replenish the liquid volume to the corresponding orifice.

[0137] like Figure 11 As shown, Figure 11 This application provides a machine vision-based device for detecting the volume of trace liquids, comprising:

[0138] Image acquisition module 01 is configured to acquire an image of a target plate containing liquid, wherein the target plate has a plurality of holes;

[0139] Processing module 02 is configured to recognize the image to segment the liquid surface in the hole and calculate the area of ​​the actual liquid surface;

[0140] The judgment module 03 is configured to convert the area of ​​the actual liquid surface into the actual volume based on a preset calibration relationship, compare the actual volume with the preset target volume to obtain the volume deviation, and send the volume deviation as communication data to the control system.

[0141] like Figure 12 As shown, Figure 12 This application provides another machine vision-based device for detecting the volume of trace liquids, comprising:

[0142] Image acquisition module 01 is configured to acquire an image of a target plate containing liquid, wherein the target plate has a plurality of holes;

[0143] Processing module 02 is configured to recognize the image to segment the liquid surface in the hole and calculate the area of ​​the actual liquid surface;

[0144] The judgment module 03 is configured to convert the area of ​​the actual liquid surface into the actual volume based on a preset calibration relationship, compare the actual volume with the preset target volume to obtain the volume deviation, and send the volume deviation as communication data to the control system.

[0145] The execution module 04 is configured to, when the volume deviation exceeds the set range, have the control system plan a replenishment path based on the volume deviation and perform a replenishment operation on the target plate according to the replenishment path.

[0146] Furthermore, one specific implementation of the image acquisition module 01 may include a camera, a telecentric lens, and an LED coaxial light source that are vertically fixed above the straight section of the circular magnetic levitation track. The camera, telecentric lens, and LED coaxial light source cooperate with each other to acquire the original full-board image of the target board.

[0147] like Figure 13 As shown, Figure 13 This is a schematic diagram of the image acquisition module 01. Furthermore, one specific implementation of the image acquisition module 01 may include:

[0148] The irradiation unit 011 is configured to irradiate the target plate using a light processing component, causing the light irradiated on the inclined hole wall of the hole to be scattered.

[0149] The transmitting unit 012 causes the light illuminating the liquid surface to be reflected vertically back to the camera in order to eliminate the reflection interference from the inclined hole wall and to acquire the image through the camera.

[0150] Drive unit 013 is configured to drive the target board to move to the inspection station through the control system;

[0151] Trigger unit 014 is configured to trigger the camera to acquire images when the target board moves to the detection station, and to obtain an original full-board image containing the hole by adjusting at least one of the camera's exposure time, gain, and white balance parameters.

[0152] It should be noted that the camera in this embodiment can be an industrial camera, which is a key component of the machine vision system. It refers to an image acquisition device that uses the photoelectric conversion principle to convert optical images into electrical signals.

[0153] It should be noted that the telecentric lens in this embodiment is a special optical lens in which the principal ray is parallel to the lens optical axis. Depending on the design, it is divided into object-side telecentric lenses, image-side telecentric lenses, and bilateral telecentric lenses.

[0154] It should be noted that the coaxial light source in this embodiment is an illumination system that uses a semi-reflective and semi-transparent beam splitter to make the light emitted by the light source be completely aligned with the imaging optical axis of the camera at 180° and directed towards the object.

[0155] In some embodiments, by integrating an industrial camera, dual telecentric lenses, and a coaxial light source into a coaxial optical path, and utilizing the different reflection angles of coaxial perpendicularly incident light on the inclined aperture wall and the horizontal liquid surface, combined with the parallel light receiving characteristics of the dual telecentric lenses, lateral reflection noise and perspective parallax are filtered and eliminated at the physical optics level. This collaborative work at the optical level provides a high-contrast, proportionally scaled original image for subsequent algorithmic data calculations, thereby enabling non-contact precision measurement of minute volumes.

[0156] The technical principle can be further explained as follows: A coaxial light source directs parallel light rays perpendicularly into the conical deep hole of the orifice plate. Because the hole wall is inclined (V-shaped or U-shaped bottom), the light rays are laterally scattered after hitting the hole wall and cannot pass through the parallel light receiving channel of the dual telecentric lens, resulting in a deep black shadow in the image; while the light rays hitting the horizontal liquid surface are vertically reflected and completely received by the dual telecentric lens, appearing as a bright white area in the image.

[0157] The specific implementation of the processing module 02 can be: an image processing computer equipped with a vision platform, which is electrically connected to the camera via a network cable, and is used to perform noise reduction processing, physical distortion correction processing, geometric positioning processing, and Canny algorithm composite segmentation on the original full plate image to obtain the actual liquid surface area, and calculate the actual volume and volume deviation based on the calibration relationship.

[0158] It should be noted that the Gaussian filtering process in this implementation can be achieved by convolving a Gaussian matrix template with the neighborhood of each pixel in the original image, and smoothing the image by weighted averaging to filter out high-frequency noise.

[0159] It should be noted that the implementation method in this case uses physical distortion correction processing, which is an algorithm that eliminates radial distortion (barrel or pincushion) and tangential distortion caused by physical manufacturing defects of the optical lens through mathematical projection transformation, so that the geometric features in the image return to their true physical form.

[0160] It should be noted that the geometric positioning process used in this implementation involves taking pictures with an industrial camera, and then the algorithm first locates the four outer geometric vertices of the perforated plate, calculates the current translation vector and rotation matrix, and corrects and aligns the entire plate image.

[0161] It should be noted that the Canny algorithm implemented in this case is a composite closed image segmentation algorithm that combines the Canny edge extraction algorithm with the region growing algorithm. Specifically, the Canny operator is used to extract the high-contrast closed circular dark edge at the junction of the liquid surface crescent and the inclined hole wall; a "seed point" is set at the center of the circle inside the circular edge; starting from this seed point, adjacent pixels with a gray level difference ≤15 are merged; once the dark edge boundary extracted by Canny is encountered, the growth stops immediately, thereby segmenting the complete circular region of the liquid surface.

[0162] When applying the above algorithms, there are often small air bubbles and surface reflections within the micro-liquid surface, creating noise. Using the Canny algorithm alone would result in numerous fragmented small rings; using region growing alone would cause the micro-liquid to leak through reflective gaps. Combining the two, using the Canny edge extraction algorithm and the region growing algorithm together, allows for precise segmentation of the actual projected area of ​​the liquid surface.

[0163] Preferably, the camera is an industrial camera.

[0164] like Figure 14 As shown, Figure 14 Is it like this? Figure 13 The diagram shows the structure of the processing module 02. Further, one specific embodiment of the processing module 02 may include:

[0165] Preprocessing unit 21 preprocesses the acquired image, wherein the preprocessing method is at least one of Gaussian filtering for noise reduction, histogram equalization, and distortion correction.

[0166] The identification unit 22 uses an algorithm to identify the center coordinates and radius of each hole.

[0167] Setting unit 23 sets each of the holes after positioning as the search area;

[0168] Extraction unit 24: Extract the initial annular edge where the meniscus intersects with the borehole wall;

[0169] The extension unit 25 extends outward from the center pixel inside the initial annular edge to the initial annular edge to segment out a circular area of ​​the liquid surface.

[0170] The recognition unit 22 can employ either a template matching algorithm or a Hough circle detection algorithm. The template matching algorithm is a target localization method based on image similarity measurement. Its basic principle is as follows: a template image containing a circular pattern of the target is pre-constructed; then, the template is traversed pixel-by-pixel across the image to be detected; at each position, the similarity between the template and the corresponding image sub-region is calculated (e.g., normalized cross-correlation, sum of squared differences, etc.); and the position with the highest similarity is determined as the location of the target.

[0171] The Hough circle detection algorithm is a geometric shape detection method based on parameter space voting. Its basic principle is as follows: First, edge detection operators (such as the Canny operator) are used to extract edge points in the image. Then, for each edge point, voting is performed along its gradient direction in a predefined circle center parameter space. The parameter with the highest number of votes is selected as the candidate circle center. Finally, for each candidate circle center, voting is performed in the radius dimension to determine the radius. This method has good robustness to partial occlusion and illumination changes of the circle, and it does not require prior knowledge of the circle's precise dimensions.

[0172] In this application, "meniscus" refers to the curved liquid surface formed by a liquid near a solid surface or container wall due to surface tension.

[0173] like Figure 15 As shown, Figure 15 Is it like this? Figure 13 A schematic diagram of the structure of the processing module 02 according to another embodiment is shown. Further, one specific implementation of the processing module 02 may include:

[0174] Calculation unit 26 is configured to calculate the number of pixels contained in the segmented circular region of the liquid surface;

[0175] Conversion unit 27 is configured to calculate the area of ​​the actual liquid surface using a calibrated conversion coefficient according to the following formula:

[0176] S = N × C;

[0177] Where S is the area of ​​the actual liquid surface, in mm. 2 N represents the number of pixels; C represents the pixel physical size conversion factor, in mm. 2 / pixel.

[0178] like Figure 16 As shown, Figure 16 Is it like this? Figure 13 The diagram shows the structure of the judgment module 03. Further, one specific implementation of the judgment module 03 may include:

[0179] Fitting unit 31 is configured to substitute the area of ​​the actual liquid surface into a preset volume and area fitting curve for calculation to obtain the actual volume, wherein the fitting curve satisfies the following formula:

[0180] V=a*S 3 +b*S 2 +c*S+d;

[0181] Where V is the actual volume, in μL; and S is the area of ​​the actual liquid surface, in mm. 2 a, b, c, and d are the fitting coefficients.

[0182] Deviation rate unit 32 is configured to calculate the volume deviation rate as the volume deviation according to the following formula:

[0183] δ=(V−V target ) / V target ×100%

[0184] Wherein, δ is the volume deviation rate; V is the actual volume; V target The preset target volume;

[0185] The communication unit 33 is configured to encapsulate at least one of the following parameters as communication data: the coordinates of the hole position, the actual volume, the preset target volume, and the volume deviation rate, and send the communication data to the control system.

[0186] Preferably, the coordinates, actual volume, preset target volume, and volume deviation rate of the hole are all packaged and packaged as communication data, and then the communication data is sent to the control system.

[0187] The format of the packaged product can be JSON or XML.

[0188] The communication protocol used to send the communication data to the control system can be TCP / IP or MQTT.

[0189] The control system can be a host computer, and the specific implementation of the host computer can be a MES control system (Manufacturing Execution System) or a PLC control system (Programmable Logic Controller). There are many similar control systems, which will not be listed one by one.

[0190] like Figure 17 As shown, Figure 17 This is a schematic diagram of the structure of the execution module 04 provided in this application. Further, one specific embodiment of the execution module 04 may include:

[0191] The marking unit 041 is configured to determine the pipetting valve island number, pipetting channel number, and the identification mark of the magnetically levitated actuator carrying the target plate based on the replenishment path.

[0192] Motion unit 042 is configured to control the magnetic levitation mover to move along the magnetic levitation track in order to transport the hole in the target plate that needs to be replenished to the direct below the corresponding liquid transfer channel;

[0193] The replenishment unit 043 is configured to control the translation and alignment of the pipetting valve island and drive the corresponding pipetting channel to perform injection, so as to replenish the amount of liquid to the corresponding orifice.

[0194] It should be noted that the target board has a number of holes. The number of holes is not specifically limited here. It can be determined according to actual needs during daily manufacturing. Generally, 384 holes can be used.

[0195] like Figure 18 As shown, Figure 18 This application provides a machine vision-based system for detecting the volume of trace liquids, comprising:

[0196] Hardware layer 091 includes at least a portion of the image acquisition module;

[0197] The image acquisition module includes at least one of a coaxial light source, a telecentric lens, and a camera, as well as a target plate containing a plurality of holes, wherein the holes contain liquid.

[0198] Algorithm layer 092 includes at least a portion of the processing module; algorithm layer 092 is configured to recognize the image to segment the liquid surface in the orifice and calculate the area of ​​the actual liquid surface;

[0199] Data layer 093; it includes at least a portion of the judgment module; data layer 093 is configured to convert the area of ​​the actual liquid surface into the actual volume based on a preset calibration relationship, compare the actual volume with a preset target volume to obtain the volume deviation, and send the volume deviation as communication data to the control layer;

[0200] Control layer 094; it includes at least a portion of the execution module; control layer 094 is configured to plan a liquid replenishment path based on the volume deviation when the volume deviation exceeds a set range;

[0201] Execution layer 095; which includes at least a portion of the execution module; execution layer 095 is configured to perform a liquid replenishment operation on the target plate according to the liquid replenishment path.

[0202] It should be noted that control layer 094 can be a host computer. The specific implementation of the host computer can be a MES control system (Manufacturing Execution System) or a PLC control system (Programmable Logic Controller). There are many similar control systems, which will not be listed one by one.

[0203] It should be noted that the execution layer 095 can be at least one of a magnetic levitation actuator and a moving valve island. The magnetic levitation actuator is configured to move the target plate containing several holes, and the moving valve island is configured to inject liquid into the holes to complete the liquid replenishment operation.

[0204] In the preferred technical solution, the execution layer 095 includes both a magnetically levitated actuator and a movable valve island.

[0205] Finally, it should be noted that in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0206] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for detecting the volume of trace liquids based on machine vision, characterized in that, The method includes the following steps: Acquire an image of a target plate containing liquid, wherein the target plate has a plurality of holes; The image is identified to segment the liquid surface in the orifice and the area of ​​the actual liquid surface is calculated. Based on a preset calibration relationship, the area of ​​the actual liquid surface is converted into the actual volume, and the actual volume is compared with the preset target volume to obtain the volume deviation. The volume deviation is then sent to the control system as communication data.

2. The method for detecting the volume of trace liquids based on machine vision according to claim 1, characterized in that, The step of acquiring an image of the target plate containing liquid includes: The target plate is illuminated using a light processing component, causing the light illuminating the inclined hole wall at the hole location to be scattered. The light illuminating the liquid surface is reflected vertically back to the camera to eliminate the glare interference from the inclined orifice wall, and the image is acquired through the camera.

3. The method for detecting the volume of trace liquids based on machine vision according to claim 2, characterized in that, The step of acquiring an image of the target plate containing the liquid further includes: The control system drives the target board to move to the testing station. When the target board moves to the inspection station, the camera is triggered to acquire an image, and at least one of the camera's exposure time, gain, and white balance parameters is adjusted to obtain an original full-board image containing the hole positions.

4. The method for detecting the volume of trace liquids based on machine vision according to claim 1, characterized in that, The step of recognizing the image to segment the liquid surface in the orifice and calculating the area of ​​the actual liquid surface specifically includes: The acquired image is preprocessed, and the preprocessing method is at least one of Gaussian filtering for noise reduction, histogram equalization, or distortion correction. An algorithm is used to identify at least one of the center coordinates or radii of each of the aforementioned hole positions; Each of the holes after positioning is defined as the search area; Extract the initial annular edge where the meniscus intersects the borehole wall; Centered on the central pixel inside the initial annular edge, the process extends outward to the initial annular edge to segment out a circular region of the liquid surface. The algorithm is at least one of template matching, circle detection, boundary detection, geometric registration, or training model.

5. The method for detecting the volume of trace liquids based on machine vision according to claim 1, characterized in that, The steps for calculating the actual liquid surface area include: Calculate the number of pixels contained in the segmented liquid surface; Using the calibrated conversion factor, the area of ​​the actual liquid surface is calculated according to the following formula: S = N × C; Where S is the area of ​​the actual liquid surface, in mm. 2 N represents the number of pixels; C represents the pixel physical size conversion factor, in mm. 2 / pixel.

6. The method for detecting the volume of trace liquids based on machine vision according to claim 1, characterized in that, The step of converting the area of ​​the actual liquid surface into the actual volume based on a preset calibration relationship includes: The actual liquid surface area is substituted into a preset volume and area fitting curve for calculation to obtain the actual volume.

7. The method for detecting the volume of trace liquids based on machine vision according to claim 1, characterized in that, The step of comparing the actual volume with the preset target volume to obtain the volume deviation, and sending the volume deviation as communication data to the control system includes: The volume deviation rate, which is the volume deviation, is calculated. The communication data is encapsulated as at least one of the following parameters: borehole coordinates, actual volume, preset target volume, and volume deviation rate, and then sent to the control system.

8. The method for detecting the volume of trace liquids based on machine vision according to claim 7, characterized in that, It also includes the following steps: The control system plans a replenishment path based on the volume deviation; the step of the control system planning the replenishment path based on the volume deviation includes: Set a qualified threshold for the volume deviation rate, and determine whether the calculated volume deviation rate exceeds the qualified threshold. If the volume deviation rate exceeds the acceptable threshold, the replenishment volume is calculated as follows: The replenishment path is planned based on the replenishment volume, the physical coordinates of the target plate, and the magnetic levitation mover supporting the target plate. The target plate is replenished with liquid according to the described replenishment path.

9. The method for detecting the volume of trace liquids based on machine vision according to claim 8, characterized in that, The steps of performing the liquid replenishment operation on the target plate according to the liquid replenishment path specifically include: Based on the replenishment path, determine the pipetting valve island number, pipetting channel number, and the identification mark of the magnetically levitated actuator carrying the target plate; The magnetic levitation mover is controlled to move along the magnetic levitation track to transport the hole in the target plate that needs to be replenished to the direct below the corresponding pipetting channel; The pipetting valve island is controlled to translate and align, and the corresponding pipetting channel is driven to perform injection to replenish the liquid volume to the corresponding orifice.

10. A machine vision-based device for detecting the volume of trace liquids, characterized in that, include: An image acquisition module is configured to acquire an image of a target plate containing liquid, wherein the target plate has a plurality of holes; The processing module is configured to recognize the image to segment the liquid surface in the hole and calculate the area of ​​the actual liquid surface; The judgment module is configured to convert the area of ​​the actual liquid surface into the actual volume based on a preset calibration relationship, compare the actual volume with the preset target volume to obtain the volume deviation, and send the volume deviation as communication data to the control system.

11. The machine vision-based micro-liquid volume detection device as described in claim 10, characterized in that, Also includes: The execution module is configured to, when the volume deviation exceeds a set range, have the control system plan a replenishment path based on the volume deviation and perform a replenishment operation on the target plate according to the replenishment path.

Citation Information

Patent Citations

  • Safe updating method for information blocks of non-contact IC (integrated circuit) card

    CN104680095A