A high-precision image segmentation and size identification algorithm for a penicillin bottle

Abstract

The application discloses a high-precision image segmentation and size identification algorithm for a penicillin bottle, comprising the following steps: collecting a penicillin bottle image and performing grayscale, denoising, contrast enhancement and normalization preprocessing; loading a pre-trained SAM 3 model, and using a multi-specification penicillin bottle dataset to fine-tune and cooperatively optimize three groups of parameters, including prompt parameters, segmentation quality parameters and performance balance parameters; generating text and visual prompts and fusing input into an optimized model; after the model generates an initial segmentation mask, post-processing is performed through morphological closing operation, opening operation and edge smoothing algorithm, and a precise mask with an edge error of less than or equal to 0.05 mm is output; finally, the mask is transmitted to a robot control system for coordinate calculation of grabbing, opening and the like; the application provides a high-precision, strong-adaptation and high-real-time penicillin bottle segmentation scheme, and significantly improves the automation level and operation stability of a dispensing robot.

Description

Technical Field

[0001] This invention relates to the field of vial size recognition technology, and in particular to a high-precision image segmentation and size recognition algorithm for vials. Background Technology

[0002] Automated intravenous dispensing robots are core equipment for automating medication dispensing in clinical settings and pharmacies. During operation, they require precise acquisition of geometric parameters such as the outline of the vial, its body diameter, neck diameter, and cap diameter to drive the robot's gripper adjustment, cap opening and positioning, and medication aspiration. Existing robot systems mostly rely on manual teaching or mechanical limiting methods to obtain dimensions, resulting in poor flexibility and low efficiency.

[0003] To enhance automatic recognition capabilities, some solutions incorporate visual technologies. For example, patent CN117224394A discloses an automated vial dispensing device and method based on a collaborative robot, which uses a visual correction module to correct the needle tip position. However, this method only compensates for the position of the syringe needle and does not involve fine segmentation and dimensional measurement of the vial body and its key parts such as the aluminum cap and neck, thus failing to provide complete guidance information for the robot's grasping and opening operations. Other medication dispensing system patents, such as CN112545888B, CN114246796B, and CN212166144U, focus primarily on mechanical structure design and system process optimization, lacking attention to deep learning-based image segmentation algorithms.

[0004] Traditional image segmentation algorithms and conventional deep learning segmentation models are easily affected by factors such as occlusion by the robot gripper, reflections from the bottle and aluminum cap, and background clutter in the drug dispensing environment. The segmented contours are often incomplete or have rough edges, leading to significant deviations in subsequent dimensional measurements and making it difficult to meet the accuracy requirement of ≤0.2mm. While the new generation general segmentation model SAM 3 boasts powerful zero-sample segmentation capabilities and edge extraction accuracy, its direct application in drug dispensing scenarios has significant shortcomings: First, it is not optimized for small, critical parts such as the aluminum cap and bottle neck, resulting in unclear segmentation and a lack of dimensional measurement benchmarks for these areas; second, the segmentation mask edges contain burrs, affecting the accuracy of dimensional calculations; third, the model is not integrated with the real-time drug dispensing process and cannot directly output coordinates for robot-executed cap opening and injection; fourth, it lacks specific adaptation for the core requirement of bottle cap opening. Therefore, a method for deep adaptation and optimization of SAM 3 is needed to meet the high-precision, real-time, and collaborative control requirements of automated intravenous injection drug dispensing robots. Summary of the Invention

[0005] This invention addresses the problem that dispensing robots cannot meet the requirements for size measurement and positioning accuracy in grasping and opening actions. It proposes a high-precision image segmentation and size recognition algorithm for vials. By designing the parameter configuration, post-processing flow, and data linkage with the robot control system of the SAM 3 model based on the geometric features of vials and the robot dispensing scenario, it achieves the beneficial effects of high segmentation accuracy, strong adaptability, fast processing speed, and direct support for precise robot operation.

[0006] To achieve the above objectives, the following technical solutions are proposed: A high-precision image segmentation and size recognition algorithm for vials includes the following steps: Step 1: Vial image acquisition and preprocessing. Acquire the original image of the vial at the dispensing robot's gripping station, and preprocess the original image to obtain a preprocessed image. Step 2: SAM 3 model initialization and parameter optimization. Load the pre-trained SAM 3 basic model, and fine-tune and optimize the parameters of the SAM 3 basic model based on the bottle image dataset to obtain the optimized SAM 3 model. Step 3: Multimodal cue generation and input, generating textual and / or visual cues to guide segmentation, and inputting the cues into the optimized SAM 3 model; Step 4: High-precision segmentation of the vial. The optimized SAM 3 model generates and optimizes the segmentation mask of the vial body based on the preprocessed image and the prompt, and outputs the segmentation mask.

[0007] Through the above technical solutions, an integrated image acquisition, model optimization, and high-precision segmentation process was designed. Specific lighting, resolution, and preprocessing methods eliminated reflections and background interference, ensuring the quality of the input image. By performing task-specific fine-tuning and parameter optimization on the general SAM 3 model, the model can focus on the key size measurement areas of the vial, thus providing the robot with a continuous, accurate, and smooth-edged vial contour mask. This directly solves technical problems such as unstable gripping and cap opening failures caused by inaccurate segmentation.

[0008] Preferably, step 2, which optimizes the parameters of the SAM 3 model, specifically includes configuring at least one set of the following parameters: Optimization of prompt parameters: Set the point prompt confidence threshold point_score_threshold to 0.5-0.6, and the box non-maximum suppression threshold box_nms_threshold to 0.55-0.6; Segmentation quality parameter optimization: Set the predicted intersection-over-union threshold pred_iou_thresh to 0.92-0.95, the stability score threshold stability_score_thresh to 0.94-0.97, and the minimum mask region area min_mask_region_area to 60-100; Performance balancing parameter optimization: Set the number of sampling points per side (points_per_side) to 52-64, the number of cropping layers (crop_n_layers) to 1, and the cropping overlap ratio (crop_overlap_ratio) to 0.35-0.4.

[0009] By using the above technical solutions, the model can more sensitively capture small structures such as aluminum caps and bottlenecks while maintaining efficient inference. At the same time, it can effectively suppress false detections caused by overlapping prompts and missegmentation caused by background noise, thereby improving the integrity and edge smoothness of the segmentation mask.

[0010] Preferably, step 2 involves parameter optimization of the SAM 3 model, including the configuration of the prompt parameter optimization, the segmentation quality parameter optimization, and the performance balance parameter optimization.

[0011] Through the above technical solutions, the behavior of SAM 3 is comprehensively constrained from three dimensions: prompt response, mask quality, and processing scale, due to the synergistic effect of the three sets of parameters. This enables the model to stably output high-quality segmentation results when dealing with vials of various sizes, lighting conditions, and cap types, ensuring the robustness and generalization ability of the algorithm.

[0012] Preferably, step 4, which generates and optimizes the segmentation mask for the vial body, specifically includes: The optimized SAM 3 model generates the initial segmentation mask; The initial segmentation mask is post-processed, which includes: filling mask holes with morphological closing operations; removing small-area noise with morphological opening operations; and optimizing mask edges with an edge smoothing algorithm so that the segmentation contour matches the actual contour of the vial.

[0013] By using the above technical solutions, a post-processing module is added on top of the deep learning model output. Morphological closing operations are used to fill in the tiny holes that may appear inside the bottle, opening operations are used to remove isolated noise, and finally, edge smoothing algorithms are used to correct the burrs and discontinuities in the mask contour. This greatly improves the geometric restoration of the mask and ensures the reliability of the benchmark for subsequent size measurements.

[0014] Preferably, the morphological closing operation uses a 3×3 kernel, and the morphological opening operation uses a 2×2 kernel; the edge smoothing algorithm includes performing Gaussian filtering on the initial segmentation mask, followed by adaptive thresholding for binarization.

[0015] By employing the above technical solutions, and by setting up small-sized processing operators and smoothing processes, the geometric accuracy of subtle features such as bottle mouth, neck, and aluminum cap edges is preserved to the greatest extent while eliminating mask defects. This avoids contour deformation that may be caused by morphological processing itself, and achieves sub-pixel level edge optimization.

[0016] Preferably, the post-processed segmentation mask has an edge smoothness error of ≤0.05mm.

[0017] Through the above technical solutions, by setting clear quantitative indicators, the output segmentation mask is not only visually smooth, but also meets the high-precision requirements for robot opening and positioning on a physical scale. Thus, the visual measurement results can be directly applied to the motion planning of the robotic arm, eliminating the intermediate manual verification step.

[0018] Preferably, the vial image acquisition and preprocessing in step 1 specifically includes: Image acquisition: An industrial camera is installed directly above the robot's gripping station, using a diffused light source of 800-1200 lux, to acquire original images of the vial at a resolution of 5120×5120 pixels. The original image contains the complete vial body and the background is pure black. Image preprocessing: The original image is sequentially subjected to grayscale conversion, Gaussian filtering for noise reduction, adaptive histogram equalization to enhance contrast, and image normalization to obtain the preprocessed image.

[0019] By using the above technical solutions, the reflection phenomenon of the bottle body and aluminum cap can be effectively suppressed and the bottle outline can be highlighted by limiting the specific range of illumination intensity, high resolution and solid color background. The preprocessing chain consisting of grayscale, noise reduction, contrast enhancement and normalization significantly improves the visual saliency of feature regions and the consistency of image quality, providing stable and low noise input for model segmentation and reducing the impact of environmental fluctuations on the algorithm.

[0020] Preferably, the grayscale conversion uses a weighted average method, and the calculation formula is: Gray = 0.299×R + 0.587×G + 0.114×B; the Gaussian filter kernel size used for denoising is 3×3, and the standard deviation σ is 1.0-1.2.

[0021] The above technical solutions employ weighted average grayscale, which conforms to the visual characteristics of the human eye, to preserve the brightness information for contour recognition to the greatest extent. Meanwhile, Gaussian filtering with specific parameters smooths noise and suppresses fine reflections without blurring the edge gradient at the junction of the bottle cap and the bottle body, enabling subsequent segmentation and edge extraction to obtain clear feature boundaries.

[0022] Preferably, in step 2, the SAM 3 basic model is fine-tuned based on the vial image dataset. Specifically, the vial image dataset contains 8,000 sample images of vials of various sizes with diameters ranging from 8mm to 30mm. A stochastic gradient descent optimizer is used, with a learning rate set to 2e-5 to 4e-5, 60-90 iterations, and a batch size set to 10-14.

[0023] By employing the above technical solutions, a dataset covering commonly used drug dispensing specifications is constructed, and fine-tuning is performed using a small learning rate and sufficient iterations. This allows the model to accurately converge to the vial segmentation task during training without catastrophic forgetting. At the same time, the appropriate batch size ensures training stability while fully utilizing the GPU's memory and computing resources, thereby improving the model's ability to discriminate vial features.

[0024] Preferably, the segmentation mask output in step 4 is directly transmitted to the coordinate transformation control system of the automated intravenous injection dispensing robot for calculating the coordinates of robot grasping, opening the cap, injecting, and withdrawing.

[0025] By directly integrating the output of the visual algorithm with the robot's motion control system, a real-time closed loop from perception to execution is formed. This allows the robot to automatically adjust its grasping posture, opening path, and injection depth based on the real-time calculated outline and size of the vial, eliminating the need for manual measurement and teaching, and improving the efficiency, consistency, and safety of medication dispensing.

[0026] The beneficial effects of this invention are: 1. The segmentation results are accurate and meet the needs of automated intravenous injection dispensing robots for dispensing vials. This invention can accurately segment partially obscured vials and directly provide accurate contour data for the robot to grasp, open, and dispense the medication, solving the problem of insufficient accuracy caused by the reliance on manual measurement for size measurement in the existing technology. 2. Highly adaptable and suitable for vial and robotic dispensing scenarios, the algorithm is optimized for the workstation environment, feature occlusion, vial specifications (8mm-30mm), different cap types, and real-time requirements of automated intravenous dispensing robots. It can adapt to scenarios such as normal workshop lighting, vials of different colors, and mixed storage of multiple specifications. There is no need to retrain the model for different scenarios. It can be seamlessly integrated into the automated intravenous dispensing robot system, reducing system integration costs. 3. High real-time performance, meeting the needs of efficient robot operation and cap opening. The optimized SAM 3 model retains the advantages of efficient inference. On a single NVIDIA 4080 GPU, the total segmentation time of a single vial image is ≤60ms, which is synchronized with the robot's grasping and cap opening frequency, ensuring the efficient operation of the robot in dispensing and opening medicines and improving dispensing efficiency. 4. Highly practical, improving the safety and stability of robot medication dispensing. The segmentation mask output by the algorithm can be used to measure the robot's dimensional parameters and to automatically adjust the robot's gripper. The cap opening positioning information can guide the robot to accurately execute the cap opening action, avoiding problems such as unstable gripping, cap opening failure, and drug spillage, reducing the cost of manual intervention, and improving the safety and stability of intravenous injection medication dispensing. 5. It has good scalability. The algorithm is built on the open-source SAM 3 model. It can further optimize the size recognition parameters, cap opening positioning parameters and expand the segmentation targets, such as the scale line on the bottle mouth, according to the needs of different models of automated intravenous injection dispensing robots. It can adapt to more robot dispensing and cap opening scenarios. At the same time, the accuracy of segmentation, size recognition and cap opening positioning can be continuously improved by continuously adding sample data. Attached Figure Description

[0027] Figure 1 This is the overall flowchart of the algorithm of the present invention, adapted to an automated intravenous injection dispensing robot system.

[0028] Figure 2 This is the original image of the vial in Embodiment 1 of the present invention.

[0029] Figure 3 This is a preprocessed image of a vial in Embodiment 1 of the present invention.

[0030] Figure 4 This is a schematic diagram of the multimodal prompt box in Embodiment 1 of the present invention.

[0031] Figure 5 This is a diagram showing the final segmentation result in Embodiment 1 of the present invention, including the segmented vial body. Detailed Implementation

[0032] The implementation of the present invention will be further described in detail below with reference to specific embodiments, but the scope of protection of the present invention is not limited thereto. In the following embodiments, the automated intravenous injection dispensing robot system developed by Meilan (Hangzhou) Pharmaceutical Technology Co., Ltd., model ML-300, is used, and the GPU used for testing is NVIDIA GeForce RTX 4080. Example

[0033] This embodiment performs high-precision image segmentation on a transparent vial with a diameter of 8mm. The vial is a pull-top type, and its actual dimensions are: body diameter 8mm, body height 35mm, neck diameter 4mm, and cap diameter 6mm.

[0034] This embodiment proposes a high-precision image segmentation and size recognition algorithm for vials, referencing... Figure 1 This includes the following steps: Step 1: Image acquisition and preprocessing of vials.

[0035] Step 101: Image Acquisition. A Hikvision industrial CCD camera is installed above the robot's gripping station, with a lens focal length set to 35mm, ensuring the camera's optical axis is perpendicular to the center line of the vial. A diffused light source is used for frontal auxiliary illumination, with the light intensity controlled at 1000 lux via a DC regulated power supply to avoid directional reflections from the vial's glass and aluminum cap, preventing the formation of bright spots. The camera's acquisition resolution is set to 5120×5120 pixels, with a frame rate of 12 frames per second, synchronized with the gripping cycle signal output by the robot controller. The vial is placed on a pure black platform where the robot gripper is located, creating a uniform low-grayscale background to minimize background interference with the segmentation task. The acquired raw image is saved in 24-bit BMP format. (Refer to...) Figure 2 It contains the complete body of the vial.

[0036] Step 102: Image preprocessing. To eliminate interference from color information that is useless for segmentation and to enhance target features, the following operations are performed in sequence: Step 121: Grayscale conversion. The original color image is converted to a grayscale image. The grayscale value of each pixel is calculated using a weighted average method. The formula is: Gray = 0.299×R + 0.587×G + 0.114×B, where R, G, and B are the red, green, and blue channel pixel values ​​of the original image, respectively. This formula allows the grayscale image to better preserve the outline of the vial and the brightness differences between the aluminum cap and the stopper.

[0037] Step 122: Denoising. A Gaussian filter algorithm is used to smooth and denoise the grayscale image. The Gaussian filter kernel size is set to 3×3, and the standard deviation σ is set to 1.1. The filtering operation eliminates random noise introduced by the CCD sensor and slight reflections from the table surface, while maintaining the gradient intensity of the bottle edge well, ensuring the edge sharpness of subsequent segmentation.

[0038] Step 123: Contrast Enhancement. An adaptive histogram equalization algorithm is used to process the denoised grayscale image, focusing on improving the local contrast between the aluminum cap and bottle neck, the connection between the bottle neck and body, and the gap between the aluminum cap and stopper. In this way, even under low or uneven lighting conditions, the key geometric boundaries required for dimensional measurement can be clearly highlighted.

[0039] Step 124: Image normalization. This linearly maps the pixel values ​​of the enhanced image to the [0,1] interval, eliminating pixel value scale differences caused by minor fluctuations in illumination intensity between different images. This unifies the distribution range of the input data, improving the numerical stability of the model's inference. (Refer to...) Figure 3 The preprocessed vial image is obtained.

[0040] Step 2: SAM 3 model initialization and parameter optimization.

[0041] Step 201: Load the model. Load the pre-trained weights of SAM 3 from Meta's official open source. This model has a Perception Encoder visual backbone network, a text prompt encoder, a cross-attention module, and a Presence Head module. The Perception Encoder is responsible for extracting multi-scale visual features of the image, and the Presence Head module is used to suppress the generation of false regions that do not exist in the image, ensuring the reliability of the segmentation results.

[0042] Step 202: Model fine-tuning. A dedicated vial image dataset for the dispensing robot is constructed. This dataset contains 8000 sample images, covering vials of common sizes (1ml, 2ml, 5ml, 10ml) and diameters ranging from 8mm to 30mm. Sample images were acquired from various angles, under different lighting conditions, and with or without occlusion from the grippers to enhance model robustness. 7000 images from the dataset are used for fine-tuning training, and 1000 images are used for testing. Labeling is primarily done manually, supplemented by machine-aided labeling. The outline polygon of the vial is accurately drawn and converted into a binary mask as training labels.

[0043] The fine-tuning process employed a stochastic gradient descent optimizer with a learning rate of 3e-5, 80 iterations, and a batch size of 12. During fine-tuning, a combination of cross-entropy and Dice loss was used as the loss function to balance pixel-level classification accuracy with region-level overlap. Through training on the aforementioned dataset, the model gradually adapted to the unique geometry of vials, the internal texture caused by transparent or semi-transparent materials, and the metallic reflective properties of the aluminum cap, significantly improving its ability to distinguish between the vial body, neck, and key components such as the aluminum cap.

[0044] Step 203: Parameter optimization configuration. To obtain the best segmentation effect, and considering the bottle adaptation requirements, the following three sets of parameters were optimized. These parameters work together during inference to control the mask generation process: Optimization of prompt parameters: Set point_score_threshold=0.55 to control the confidence level of point prompts and ensure sensitive response to prompts for small targets such as aluminum cap areas; Set box_nms_threshold=0.58 to control the suppression level of multiple overlapping detection boxes and avoid missed detections when multiple sizes of vials are placed together.

[0045] Segmentation quality parameter optimization: setting pred_iou_thresh=0.94 requires a high confidence level in the intersection-union ratio (IU) between the model's predicted segmentation mask and the actual mask, thus ensuring that the output bottle outline closely matches the actual boundary; setting stability_score_thresh=0.96 increases the stability score threshold, ensuring that the output mask has consistent edges under different cue conditions, effectively suppressing edge jitter; setting min_mask_region_area=80 filters out small fragmented regions with an area less than 80 pixels, preventing point reflections or dust in the background from being mis-segmented as part of the vial.

[0046] Performance balancing parameter optimization: Set points_per_side=58, which is the number of points uniformly sampled on each side of the image. A higher sampling density helps to generate finer-grained masks in small areas such as bottle caps; Set crop_n_layers=1 to enable multi-scale cropping in one layer, so that the model can analyze the global outline and local details of the bottle under different views; Set crop_overlap_ratio=0.38, which is the overlap ratio between cropped regions, to ensure that key junctions such as bottle caps and bottle necks remain intact after cropping and recombination.

[0047] Step 3: Multimodal prompt generation and input.

[0048] Step 301: Text prompt generation. Based on robot size recognition and cap opening requirements, two text prompt strings are generated: "vial" and "vial body". These two prompts are in English and are directly input into the SAM 3 text prompt encoder to guide the model to focus on the entire vial and its body area.

[0049] Step 302: Visual cues are generated. Simultaneously, the YOLOv8 object detection algorithm is used to infer the bounding box coordinates of the main body of the vial from the preprocessed image, serving as the input model for the cues. This bounding box provides the model with a coarse location prior, significantly accelerating the convergence speed of the segmentation mask and improving localization accuracy.

[0050] Step 303: Multimodal cue fusion. Text cue and bounding box cue are input into the optimized SAM 3 model's cue encoder. The encoder fuses this heterogeneous information into a unified cue feature vector. This feature vector is then deeply fused with image features in the cross-attention module, accurately locating the body and neck of the vial. This provides precise guidance for subsequent high-precision segmentation. (Results referenced...) Figure 4 .

[0051] Step 4: High-precision segmentation of vials.

[0052] Step 401: Feature Extraction and Initial Segmentation. The optimized SAM 3 model performs forward inference on the preprocessed vial image using the Perception Encoder, extracting vial body, neck contour features, texture features, and semantic features at multiple scales. The cross-attention module interacts with the extracted image features and the fused cue features generated in Step 3 to highlight the response of the vial region. Subsequently, the model's mask generation head, combined with the optimized segmentation quality parameters from Step 203, predicts the initial segmentation mask for the vial body on the feature map.

[0053] Step 402: Segmentation mask optimization. Post-process the initial segmentation mask to eliminate common minor defects in the model output. The specific operations are as follows: First, morphological closing operations are employed, using 3×3 rectangular structuring elements to perform a dilation-erosion process on the mask. This step effectively fills the tiny holes inside the bottle caused by sensor noise or local uncertainties in the model, ensuring the continuity and integrity of the mask area.

[0054] Secondly, morphological opening operations are employed, using 2×2 rectangular structuring elements to perform erosion followed by dilation on the mask. This operation can remove isolated small-area noise on the outer side of the bottle or in the background, which may be misidentified by the model as part of the vial, resulting in a cleaner outline.

[0055] Finally, an edge smoothing algorithm is applied. This algorithm first smooths the mask using a 3×3 Gaussian filter kernel with a standard deviation of 0.5, resulting in a smooth, sloping edge. Then, an adaptive thresholding method is used to automatically determine the binarization threshold based on the local grayscale distribution of the mask edges, resharpening the sloping edges into clear, smooth boundary lines. The results are referenced below. Figure 5 After this processing, the edges of the segmentation mask closely match the actual physical contour of the vial. Verification was performed using a high-precision digital microscope, and the edge smoothness error was controlled within 0.05mm.

[0056] Step 403: Result Output and Linkage. The optimized segmentation mask, along with the extracted geometric parameters such as bottle diameter, neck diameter, aluminum cap diameter, and bottle height, is directly transmitted to the coordinate transformation control system of the ML-300 dispensing robot via Ethernet protocol. The control system calculates the spatial position and orientation of the vial in the robot's base coordinate system based on the segmentation mask, and further generates the adjustment amount of the gripper, the positioning coordinates of the cap-opening claw, and the puncture point coordinates of the needle tip during injection and aspiration. The entire process, from image acquisition to coordinate calculation, takes no more than 60ms for segmentation of a single image, meeting real-time requirements.

[0057] Step 5: Result verification and feedback optimization.

[0058] Step 501: The segmentation results of this embodiment are quantitatively evaluated using common evaluation metrics for semantic segmentation. Compared with the standard mask annotated manually, the intersection-union ratio (IU / UK) of the segmentation mask in this embodiment reaches 0.94, and the Dice coefficient reaches 0.96. The output mask edges are continuous and smooth, without burrs or abrupt gaps. When the mask outline is superimposed on the original image, the outline completely coincides with the edge of the actual vial. The vial size calculated based on the mask is compared with the actual size, and the error is within 0.15mm, meeting the accuracy requirements for robot grasping and opening.

[0059] Step 502: Add the acquired image and its high-quality segmentation mask as new training data to the model training set for continuous iterative updates to the model, so as to continuously optimize the model's generalization ability and segmentation accuracy on different sizes and batches of vials. Example

[0060] This embodiment performs image segmentation on a labeled vial with a diameter of 12mm. This vial is also a pull-cap type, with actual dimensions of: body diameter 12mm, body height 50mm, neck diameter 5mm, and cap diameter 8mm. The image acquisition scene simulates a low-light workshop environment, reducing the light intensity to 800 lux to verify the algorithm's robustness under non-ideal lighting conditions.

[0061] Step 1: Image Acquisition and Preprocessing. The hardware configuration is the same as in Example 1, except that the output of the light source dimmer is adjusted so that the illuminance meter displays a value of 800 lux. Since the bottle has a paper label with printed characters, it creates new interference with the background texture. The preprocessing steps are consistent with Example 1, performing grayscale conversion, Gaussian filtering for noise reduction, adaptive histogram equalization, and normalization in sequence. Observation shows that adaptive histogram equalization plays a crucial role in improving the contrast at the junction of the bottle cap and label, as well as at the junction of the bottle neck and label; despite the printed pattern on the label, the overall outline of the bottle is still highlighted.

[0062] Step 2: Model Initialization and Parameter Optimization. The dataset and training configuration used for model fine-tuning are the same as in Example 1. To address low-light conditions and label interference, the inference parameters were appropriately adjusted to achieve more robust segmentation results. The specific parameter configuration is as follows: Tip parameters: point_score_threshold=0.5, slightly lower than in Example 1, to improve tolerance for low-contrast tips in low light; box_nms_threshold=0.55, also appropriately relaxed to ensure that overlapping boxes can be preserved.

[0063] Segmentation quality parameters: pred_iou_thresh=0.92, stability_score_thresh=0.94, min_mask_region_area=60.

[0064] Under low light conditions, the confidence of model segmentation may decrease slightly. Appropriately reduce the cross-union threshold and stability threshold to ensure that effective masks are not filtered out; reduce the minimum mask area to filter out smaller noise points.

[0065] Performance balancing parameters: points_per_side=52, crop_overlap_ratio=0.35. Due to the large size of the bottle, appropriately reducing the number of sampling points and the overlap ratio will not result in a loss of coverage, but will instead reduce the amount of computation and ensure processing speed.

[0066] Step 3: Multimodal cue generation and input. Similar to Example 1, text cues "vial" and "vialbody," along with bounding box cues generated by YOLOv8, are fused and input into the model. Despite the presence of character patterns on the label, YOLOv8's bounding box detection accurately located the bottle area, unaffected by label interference.

[0067] Step 4: High-precision segmentation of the bottle. The model inference and post-processing workflow is the same as in Example 1. In the initial segmentation mask, several tiny holes appeared in the label area due to texture changes, and there were a few small noise spots near the label edges. After morphological closing operations with 3×3 kernels, these holes were completely filled; then, after morphological opening operations with 2×2 kernels, several isolated noise spots in the background were removed. The final Gaussian filtering and adaptive thresholding processing made the bottle mask edges very smooth and successfully restored the fine contours of the bottle cap and neck.

[0068] Step 5: Result Verification. The output segmentation mask is visually evaluated and compared with the original image. The segmentation mask perfectly matches the actual outline of the vial, with no gaps or redundant edges. Quantitative evaluation indicators are: segmentation intersection-union ratio (CIU) reaches 0.93, and the Dice coefficient reaches 0.95. The edge smoothness error is less than 0.05mm. The vial diameter, cap diameter, and other dimensions calculated based on this mask are compared with the measured values ​​using calipers; the maximum deviation is 0.18mm, meeting the error requirements for robotic dispensing.

[0069] This embodiment demonstrates that the method proposed in this invention can still maintain high-precision segmentation and size recognition performance under challenging conditions such as low-light workshop environments and labeled bottles, showing extremely high industrial practical value.

[0070] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A high-precision image segmentation and size recognition algorithm for vials, characterized in that, Includes the following steps: Step 1: Vial image acquisition and preprocessing. Acquire the original image of the vial at the dispensing robot's gripping station, and preprocess the original image to obtain a preprocessed image. Step 2: SAM 3 model initialization and parameter optimization. Load the pre-trained SAM 3 basic model, and fine-tune and optimize the parameters of the SAM 3 basic model based on the bottle image dataset to obtain the optimized SAM 3 model. Step 3: Multimodal cue generation and input, generating textual and / or visual cues to guide segmentation, and inputting the cues into the optimized SAM 3 model; Step 4: High-precision segmentation of vials. The optimized SAM 3 model generates and optimizes the segmentation mask of the vial body based on the preprocessed image and the prompt, and outputs the segmentation mask.

2. The high-precision image segmentation and size recognition algorithm for vials according to claim 1, characterized in that, Step 2, the parameter optimization of the SAM 3 model, specifically includes configuring at least one set of the following parameters: Optimization of prompt parameters: Set the point prompt confidence threshold to 0.5-0.6 and the bounding box non-maximum suppression threshold to 0.55-0.6; Segmentation quality parameter optimization: Set the predicted intersection-union ratio threshold to 0.92-0.95, the stability score threshold to 0.94-0.97, and the minimum mask area to 60-100. Performance balancing parameter optimization: Set the number of sampling points per side to 52-64, the number of clipping layers to 1, and the clipping overlap ratio to 0.35-0.

4.

3. The high-precision image segmentation and size recognition algorithm for vials according to claim 2, characterized in that, Step 2 involves parameter optimization of the SAM 3 model, including the configuration of the prompting parameter optimization, the segmentation quality parameter optimization, and the performance balance parameter optimization.

4. The high-precision image segmentation and size recognition algorithm for vials according to claim 1, characterized in that, Step 4, which generates and optimizes the segmentation mask for the main body of the vial, specifically includes: The optimized SAM 3 model generates the initial segmentation mask; The initial segmentation mask is post-processed, which includes: filling mask holes with morphological closing operations; removing small-area noise with morphological opening operations; and optimizing mask edges with an edge smoothing algorithm so that the segmentation contour matches the actual contour of the vial.

5. The high-precision image segmentation and size recognition algorithm for vials according to claim 4, characterized in that, The morphological closing operation uses a 3×3 kernel, and the morphological opening operation uses a 2×2 kernel; the edge smoothing algorithm includes performing Gaussian filtering on the initial segmentation mask, followed by adaptive thresholding for binarization.

6. The high-precision image segmentation and size recognition algorithm for vials according to claim 4, characterized in that, The post-processed segmentation mask has an edge smoothness error of ≤0.05mm.

7. The high-precision image segmentation and size recognition algorithm for vials according to claim 1, characterized in that, The vial image acquisition and preprocessing in step 1 specifically includes: Image acquisition: An industrial camera is installed directly above the robot's gripping station, using a diffused light source of 800-1200 lux, to acquire original images of the vial at a resolution of 5120×5120 pixels. The original image contains the complete vial body and the background is pure black. Image preprocessing: The original image is sequentially subjected to grayscale conversion, Gaussian filtering for noise reduction, adaptive histogram equalization to enhance contrast, and image normalization to obtain the preprocessed image.

8. The high-precision image segmentation and size recognition algorithm for vials according to claim 7, characterized in that, The grayscale conversion uses a weighted average method, and the calculation formula is: Gray = 0.299×R + 0.587×G + 0.114×B; the denoising uses a Gaussian filter kernel size of 3×3 and a standard deviation σ of 1.0-1.

2.

9. The high-precision image segmentation and size recognition algorithm for vials according to claim 1, characterized in that, Step 2 involves fine-tuning the SAM 3 basic model based on the vial image dataset. Specifically, the vial image dataset contains 8,000 sample images of vials of various sizes with diameters ranging from 8mm to 30mm. A stochastic gradient descent optimizer is used, with a learning rate set to 2e-5 to 4e-5, 60-90 iterations, and a batch size set to 10-14.

10. The high-precision image segmentation and size recognition algorithm for vials according to claim 1, characterized in that, The segmentation mask output in step 4 is directly transmitted to the coordinate transformation control system of the automated intravenous injection dispensing robot for calculating the coordinates of robot grasping, opening the cap, injecting, and withdrawing.

Images