Intelligent detection method of biomedical test tube

Abstract

The invention discloses an intelligent detection method for biomedical test tubes, and belongs to the technical field of biological test tube detection.The intelligent detection method comprises the steps that image collection is conducted on different types of test tubes, and a YOLO model is used for conducting marking type training; the trained YOLO model is converted into a standardized cross-platform model format through an ONNX framework, and edge deployment reasoning is carried out; carrying out system calibration and initial reference coordinate establishment; collecting a current state image of the to-be-detected test tube box, and performing anomaly detection; performing test tube coordinate analysis and structured positioning on the image passing the anomaly detection; and the test tube space position relation obtained through detection and matching is mapped into a mechanical arm coordinate system, and the target test tube is grabbed. According to the method, the abnormal state in the test tube grabbing process can be automatically recognized and analyzed under the severe condition, the position of a grabbing target is accurately positioned, a mechanical arm is effectively guided to complete grabbing and releasing actions, and the operation reliability of a test tube carrying system is improved.

Description

Technical Field

[0001] This invention relates to the field of biological test tube testing technology, and in particular to an intelligent testing method for biomedical test tubes. Background Technology

[0002] In modern biomedical research, chemical experiments, and related industrial production processes, test tubes are widely used as common experimental equipment for sample preservation, reaction operations, and analysis. With the expansion of experimental scale and the improvement of automation levels, how to efficiently and accurately complete the task of transporting test tubes has become one of the key issues in ensuring experimental efficiency and sample safety. Especially in some high-throughput experimental scenarios, the need to frequently transfer test tubes from one test tube box to another is very common. However, traditional manual handling methods are not only labor-intensive and inefficient, but also prone to operational errors due to fatigue or negligence, thus affecting the reliability of experimental results. Currently, some laboratories have tried to introduce semi-automated equipment to assist in test tube handling, but these methods still have significant limitations. For example, the equipment has poor adaptability and cannot cope with changes in test tube box sizes; or it relies on preset paths and cannot flexibly handle unexpected situations such as test tube displacement or damage. Furthermore, in some special experimental environments, such as low temperature, high humidity, sterile rooms, or low light environments, traditional equipment often struggles to operate stably, further limiting its application scope.

[0003] Traditional test tube handling systems often suffer from insufficient accuracy and stability when performing test tube grasping and placement operations, especially in extreme experimental environments. Lacking real-time sensing capabilities of the test tube's status, existing systems struggle to promptly identify anomalies during the grasping process, such as test tube misalignment, tilting, breakage, or grasping failure, thus impacting overall handling efficiency and reliability. Furthermore, the movement of robotic arms typically relies on pre-programmed sequences, lacking dynamic adjustment capabilities and failing to effectively cope with changes and uncertainties in real-world scenarios. Therefore, existing methods are insufficient to meet the demands of modern laboratories and industrial production for high-precision, highly intelligent test tube handling operations. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent detection method for biomedical test tubes, which combines anomaly detection with robotic arm guidance. Under low temperature and low light conditions, it can automatically identify and analyze abnormal states during the test tube grasping process, and accurately locate the spatial position of the grasping target. This effectively guides the robotic arm to complete the accurate grasping and stable release of the test tube, significantly improving the intelligence level and operational reliability of the test tube handling system.

[0005] To achieve the above objectives, the present invention provides an intelligent detection method for biomedical test tubes, comprising the following steps:

[0006] S1. Acquire images of different types of test tubes and perform labeled training using the YOLO model;

[0007] S2. Convert the trained YOLO model into a standardized cross-platform model format using the ONNX framework and perform edge deployment inference.

[0008] S3. Perform system calibration and establish initial reference coordinates;

[0009] S4. Collect the current status image of the test tube box to be inspected, and transmit the image data to the deployed YOLO model for anomaly detection;

[0010] S5. Perform test tube coordinate analysis and structured localization on images that pass anomaly detection;

[0011] S6. Map the spatial position relationship of the test tube obtained from detection and matching to the coordinate system of the robotic arm, and feed it back to the grasping control system to accurately grasp and stably release the target test tube.

[0012] Preferably, S1 specifically includes the following steps:

[0013] S11. An industrial-grade camera with antifreeze optical packaging and infrared supplementary lighting module is used to acquire images of biomedical test tubes of different models, materials and placement conditions to build a representative training sample set. The minimum temperature of the acquisition environment is -20℃ and the minimum light intensity is below 25 lux. The images include the normal arrangement of test tubes and common abnormal shapes such as tilting, displacement and missing caps to enhance the robustness of the model.

[0014] S12. Low-light image enhancement algorithm, frosting edge reconstruction algorithm and multi-frame fusion noise reduction mechanism are introduced to restore image quality;

[0015] S13. The repaired image is semantically annotated by an AI-assisted annotation system and used as a training sample to input into the YOLO detection model. The YOLO detection model adopts the CSPDarkNet backbone network, PANet path aggregation network and SPP spatial pyramid structure. The YOLO detection model is trained under supervision on the training samples to extract the bounding box, state, category label and spatial location information of the test tube, and obtain a deep YOLO detection model with end-to-end target recognition and localization capabilities.

[0016] Preferably, S2 specifically includes the following steps:

[0017] S21. The deep YOLO detection model is structurally transformed using the ONNX intermediate representation format and converted into a standardized cross-platform model format.

[0018] S22. Perform graph optimization and redundant node pruning on the transformed deep YOLO detection model, and combine BatchNormalization fusion and weight compression strategies to reduce memory and computing resource consumption.

[0019] S23. Deploy the deep YOLO detection model on either a non-GPU-based embedded platform or a main industrial control computer. Combine it with a lightweight inference engine to achieve efficient target recognition in a low-power environment. The lightweight inference engine includes ONNX Runtime and NCNN.

[0020] Preferably, step S3 specifically includes the following steps:

[0021] S31. The standard full-cell test tube rack is spatially photographed and calibrated by a vision module mounted on the end of a six-axis collaborative robotic arm, and the precise position of the test tube center in the world coordinate system is obtained by using the camera intrinsic and extrinsic parameter correction matrix.

[0022] S32. Construct a reference point configuration file with the precise position of the test tube center in the world coordinate system, encapsulate it as structured data, and use it for subsequent spatial matching and error judgment of actual test results.

[0023] Preferably, step S4 specifically includes the following steps:

[0024] S41. The robotic arm controls the shooting module to take top-down shots, collect images of the current status of the test tube box to be inspected, and transmit the image data to the deployed YOLO model for real-time inference.

[0025] S42. The deployed YOLO model performs object detection in the image, identifies the position and status information of each test tube, calls the status judgment logic tree to perform multiple verifications, and determines whether there are any anomalies, including but not limited to test tube tilting, missing, broken, overturned, and cap falling off. It outputs the target category, confidence score, and two-dimensional bounding box.

[0026] S43. For abnormal results, trigger on-site audible and visual alarms and operation termination through either PLC or ROS message mechanism, and feed back the abnormal image and its detection results to the host computer interface for manual confirmation and system processing.

[0027] Preferably, in S5, specifically:

[0028] S51. The deployed YOLO model outputs a list of coordinates of the center points of all test tubes in the current frame and calculates the average pixel area of ​​the detection box.

[0029] S52. Perform preliminary classification of test tube models based on pixel area to facilitate the adaptation of multiple test tube models.

[0030] S53. Call the reference point configuration file corresponding to the current test tube model, and compare and match the detected coordinates with the reference point configuration file according to the error tolerance. If the Euclidean distance between the center point of the test tube and the reference point configuration file is within the set threshold range, the test tube is determined to be located in the specified row and column position. If it exceeds the error threshold, it is considered to be misaligned.

[0031] Preferably, in S6, the grasping control system combines the visual positioning results with the coordinate transformation relationship of the robot body to calculate the motion trajectory and posture adjustment information required by the end effector of the robotic arm, so as to accurately grasp and stably release the target test tube.

[0032] Therefore, the intelligent detection method for biomedical test tubes described above possesses excellent system robustness, can adapt to and efficiently process biomedical test tubes, effectively covers diverse test tube specifications and complex scenarios, and improves the versatility and applicability of the detection system. For the extreme low-temperature environment of the test tubes, through diversified training samples and specific optimization of the YOLO detection model, the system's environmental adaptability and anti-interference performance are enhanced, effectively resisting the effects of frost, low light, and temperature fluctuations. The efficient and accurate detection capability significantly improves the level of automation, providing solid technical support for the precise grasping and operation of the robotic arm, thereby ensuring the safe management and efficient flow of biological test tubes in extreme environments.

[0033] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0034] Figure 1 This is a flowchart of an embodiment of an intelligent detection method for biomedical test tubes according to the present invention;

[0035] Figure 2 This is an actual effect diagram of an embodiment of the intelligent detection method for biomedical test tubes according to the present invention;

[0036] Figure 3 This is a diagram of the actual operation interface of Vision Master, an embodiment of the intelligent detection method for biomedical test tubes according to the present invention. Detailed Implementation

[0037] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0038] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0039] Example 1

[0040] This invention provides an intelligent detection method for biomedical test tubes, the detection process of which is as follows: Figure 1 As shown, it includes the following steps:

[0041] S1. Images of different types of test tubes are acquired and labeled using the YOLO model for training. Specifically:

[0042] S11. An industrial-grade camera equipped with anti-freeze optical packaging and infrared supplementary lighting modules is used to acquire images of biomedical test tubes of different models, materials, and placement conditions to construct a representative training sample set. The acquisition environment temperature is as low as -80℃ and the light intensity is as low as 25 lux. The differences in optical performance of test tubes in extremely cold, low light, and frost-prone environments are fully considered to ensure that the data covers abnormalities such as image distortion and contrast reduction that may be caused by environmental interference. The images include test tubes in normal arrangement and common abnormal shapes such as tilting, displacement, and missing caps to enhance the robustness of the model and construct a multi-scene, multi-category sample database.

[0043] S12. Low-light image enhancement algorithm, frosting edge reconstruction algorithm and multi-frame fusion noise reduction mechanism are introduced to restore image quality;

[0044] S13. The repaired image is semantically annotated by an AI-assisted annotation system and used as a training sample to input into the YOLO detection model. The YOLO detection model adopts the CSPDarkNet backbone network, PANet path aggregation network and SPP spatial pyramid structure. The YOLO detection model is trained under supervision on the training samples to extract the bounding box, state (such as tilt, capping, missing, broken, etc.), category label and spatial location information of the test tube, and obtain a deep YOLO detection model with end-to-end target recognition and localization capabilities.

[0045] S2. Convert the trained YOLO model into a standardized cross-platform model format using the ONNX framework, and perform edge deployment inference, specifically:

[0046] S21. To meet the real-time response and integrated deployment requirements of the detection system, the deep YOLO detection model is structurally converted through the ONNX intermediate expression format to a standardized cross-platform model format. This step ensures the compatibility of the deep YOLO detection model in cross-platform vision systems, especially suitable for industrial edge devices with limited computing power but extremely high reliability requirements.

[0047] S22. Perform graph optimization and redundant node pruning on the transformed deep YOLO detection model. Combine BatchNormalization fusion and weight compression strategies to reduce memory and computing resource consumption, ensuring long-term stable operation of the model in harsh environments such as extreme cold and frost, while ensuring optimal synergy between detection speed and accuracy.

[0048] S23. Load the deep YOLO detection model onto a non-GPU architecture embedded platform or a main industrial control computer, and combine it with a lightweight inference engine to achieve efficient target recognition in a low-power environment. The lightweight inference engine includes ONNX Runtime and NCNN. After actual testing, the detection system can stably maintain a single-frame detection speed of milliseconds (about 50ms), which meets the strict requirements of real-time performance and responsiveness on site.

[0049] S3. Perform system calibration and establish initial reference coordinates, specifically as follows:

[0050] S31. Using the vision module mounted on the end of the six-axis collaborative robotic arm, the standard full-fill test tube rack is photographed and calibrated in space, i.e. the test tubes are completely filled and there is no misalignment. The precise position of the test tube center in the world coordinate system is obtained by using the camera intrinsic and extrinsic parameter correction matrix.

[0051] S32. Construct a reference point configuration file with the precise position of the test tube center in the world coordinate system. The reference point configuration file is named center_points.pkl. Encapsulate the reference point configuration file into structured data for subsequent spatial matching and error judgment of actual test results.

[0052] S4. Acquire the current state image of the test tube box to be inspected, and transmit the image data to the deployed YOLO model for anomaly detection. Specifically:

[0053] S41. The robotic arm controls the shooting module to take top-down shots, collect images of the current status of the test tube box to be inspected, and transmit the image data to the deployed YOLO model for real-time inference.

[0054] S42. The deployed YOLO model performs object detection in the image, identifies the position and status information of each test tube, calls the status judgment logic tree to perform multiple verifications, and determines whether there are any anomalies, including but not limited to test tube tilting, missing, broken, overturned, and cap falling off. It outputs the target category, confidence score, and two-dimensional bounding box.

[0055] S43. For abnormal results, trigger on-site audible and visual alarms and operation termination through either PLC or ROS message mechanism, and feed back the abnormal image and its detection results to the host computer interface for manual confirmation and system processing, so as to avoid subsequent operational errors caused by abnormal states.

[0056] S5. Perform test tube coordinate analysis and structured localization on the images that pass anomaly detection, specifically as follows:

[0057] S51. The deployed YOLO model outputs a list of coordinates of the center points of all test tubes in the current frame and calculates the average pixel area of ​​the detection box.

[0058] S52. Perform preliminary classification of test tube models based on pixel area to facilitate the adaptation of multiple test tube models.

[0059] S53. Call the reference point configuration file corresponding to the current test tube model, and compare and match the detected coordinates with the reference point configuration file according to the error tolerance. If the Euclidean distance between the center point of the test tube and the reference point configuration file is within the set threshold range, the test tube is determined to be located in the specified row and column position. If it exceeds the error threshold, it is considered to be misaligned.

[0060] S6. The spatial positional relationship of the detected and matched test tubes is mapped to the robotic arm coordinate system and fed back to the grasping control system. The grasping control system, combining the visual positioning results with the robot's coordinate transformation relationship, calculates the motion trajectory and attitude adjustment information required by the robotic arm's end effector, enabling precise grasping and stable release of the target test tube. This process requires no human intervention and can automatically complete the identification, positioning, and handling of test tubes in an unattended state, providing strong support for efficient automated operation in low-temperature, low-light, and unmanned biological experimental environments. The actual detection effect of this method is as follows: Figure 2-3 As shown, Figure 3 This is a screenshot of an algorithm platform interface from a domestic manufacturer. The method described in this embodiment was deployed to this platform for testing, and the test results are as follows. Figure 2 As shown. The specific operation process is as follows:

[0061] The robotic arm moves to a designated location to take a picture. The image is then processed by a deployed algorithm and visual model to identify the specific position of each test tube, specifically its row and column information. This detected position information is then fed back to the robotic arm, guiding it to the designated row and column to retrieve the required test tubes and present them to the customer. The test tube position information is output in the form of 1, 1, 1; 2, 1, 0, etc., indicating that there are test tubes in the first row of the first column, and no test tubes in the first row of the second column, respectively.

[0062] Therefore, this invention employs the aforementioned intelligent detection method for biomedical test tubes, possessing excellent system robustness. It can adapt to and efficiently process biomedical test tubes, achieving fully automated, seamless, and accurate detection. It effectively covers diverse test tube specifications and complex scenarios, enhancing the versatility and applicability of the detection system. Addressing the extreme low-temperature environment of the test tubes, the system enhances its environmental adaptability and anti-interference performance through diverse training samples and specific optimization of the YOLO detection model. This effectively resists the effects of frost, low light, and temperature fluctuations, ensuring a stable and reliable detection process and avoiding misjudgments and detection failures caused by environmental factors. The detection system possesses ultra-high-speed image processing capabilities, with an average detection time of 50 milliseconds per image. Combined with precise target localization and anomaly recognition algorithms, it achieves a high overall detection accuracy. This efficient and accurate detection capability not only significantly improves the level of automation but also provides solid technical support for the precise grasping and operation of robotic arms, thereby ensuring the safe management and efficient flow of biological test tubes in extreme environments.

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. An intelligent detection method for biomedical test tubes, characterized in that, Includes the following steps: S1. Acquire images of different types of test tubes and perform labeled training using the YOLO model; S2. Convert the trained YOLO model into a standardized cross-platform model format using the ONNX framework and perform edge deployment inference. S3. Perform system calibration and establish initial reference coordinates; S4. Collect the current status image of the test tube box to be inspected, and transmit the image data to the deployed YOLO model for anomaly detection; S5. Perform test tube coordinate analysis and structured localization on images that pass anomaly detection; S6. Map the spatial position relationship of the test tube obtained from detection and matching to the coordinate system of the robotic arm, and feed it back to the grasping control system to accurately grasp and stably release the target test tube.

2. The intelligent detection method for biomedical test tubes according to claim 1, characterized in that: S1 specifically includes the following steps: S11. An industrial-grade camera with anti-freeze optical packaging and infrared supplementary lighting module is used to acquire images of biomedical test tubes of different models, materials and placement conditions to build a representative training sample set. The minimum acquisition temperature is -80℃ and the minimum light intensity is 25 lux. The images include the normal arrangement of test tubes and common abnormal shapes such as tilting, displacement and missing caps to enhance the robustness of the model. S12. Low-light image enhancement algorithm, frosting edge reconstruction algorithm and multi-frame fusion noise reduction mechanism are introduced to restore image quality; S13. The repaired image is semantically annotated by an AI-assisted annotation system and used as a training sample to input into the YOLO detection model. The YOLO detection model adopts the CSPDarkNet backbone network, PANet path aggregation network and SPP spatial pyramid structure. The YOLO detection model is trained under supervision on the training samples to extract the bounding box, state, category label and spatial location information of the test tube, and obtain a deep YOLO detection model with end-to-end target recognition and localization capabilities.

3. The intelligent detection method for biomedical test tubes according to claim 2, characterized in that: S2 specifically includes the following steps: S21. The deep YOLO detection model is structurally transformed using the ONNX intermediate representation format and converted into a standardized cross-platform model format. S22. Perform graph optimization and redundant node pruning on the transformed deep YOLO detection model, and combine BatchNormalization fusion and weight compression strategies to reduce memory and computing resource consumption. S23. Deploy the deep YOLO detection model on either a non-GPU-based embedded platform or a main industrial control computer. Combine it with a lightweight inference engine to achieve efficient target recognition in a low-power environment. The lightweight inference engine includes ONNX Runtime and NCNN.

4. The intelligent detection method for biomedical test tubes according to claim 3, characterized in that: S3 specifically includes the following steps: S31. The standard full-cell test tube rack is spatially photographed and calibrated by a vision module mounted on the end of a six-axis collaborative robotic arm, and the precise position of the test tube center in the world coordinate system is obtained by using the camera intrinsic and extrinsic parameter correction matrix. S32. Construct a reference point configuration file with the precise position of the test tube center in the world coordinate system, encapsulate it as structured data, and use it for subsequent spatial matching and error judgment of actual test results.

5. The intelligent detection method for biomedical test tubes according to claim 4, characterized in that: S4 specifically includes the following steps: S41. The robotic arm controls the shooting module to take top-down shots, collect images of the current status of the test tube box to be inspected, and transmit the image data to the deployed YOLO model for real-time inference. S42. The deployed YOLO model performs object detection in the image, identifies the position and status information of each test tube, calls the status judgment logic tree to perform multiple verifications, and determines whether there are any anomalies, including but not limited to test tube tilting, missing, broken, overturned, and cap falling off. It outputs the target category, confidence score, and two-dimensional bounding box. S43. For abnormal results, trigger on-site audible and visual alarms and operation termination through either PLC or ROS message mechanism, and feed back the abnormal image and its detection results to the host computer interface for manual confirmation and system processing.

6. The intelligent detection method for biomedical test tubes according to claim 5, characterized in that: In S5, specifically: S51. The deployed YOLO model outputs a list of coordinates of the center points of all test tubes in the current frame and calculates the average pixel area of ​​the detection box. S52. Perform preliminary classification of test tube models based on pixel area to facilitate the adaptation of multiple test tube models. S53. Call the reference point configuration file corresponding to the current test tube model, and compare and match the detected coordinates with the reference point configuration file to determine the error tolerance. If the Euclidean distance between the center point of the test tube and the reference point configuration file is within the set threshold range, the test tube is determined to be located in the specified row and column position. If it exceeds the error threshold, it is considered misaligned.

7. The intelligent detection method for biomedical test tubes according to claim 6, characterized in that: In S6, specifically: the grasping control system combines the visual positioning results with the coordinate transformation relationship of the robot body to calculate the motion trajectory and posture adjustment information required by the end effector of the robotic arm, so as to accurately grasp and stably release the target test tube.

Images