A needle tube foreign matter detection method
By employing static background modeling and multi-frame fusion technology in needle foreign object detection, the problem of false detection caused by illumination fluctuations and mechanical jitter in needle foreign object detection is solved. This achieves suppression of illumination fluctuations and mechanical jitter, reduces false alarms due to noise in a single frame, and improves the stability and accuracy of detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JIANGSU XINTIMU INTELLIGENT EQUIP CO LTD
- Filing Date
- 2026-02-11
- Publication Date
- 2026-06-05
Smart Images

Figure CN122156237A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a method for detecting foreign objects in needles, belonging to the field of machine vision and industrial inspection technology. Background Technology
[0002] In the field of industrial vision inspection, especially in the online detection of foreign objects in medical needles, traditional methods typically rely on inter-frame differencing and fixed threshold segmentation techniques. These methods exhibit poor robustness when faced with lighting fluctuations, mechanical jitter, and complex background textures in real-world production environments, easily leading to false positives and false negatives. Furthermore, using fixed regions of interest (ROIs) cannot adapt to minute changes in product position or orientation, further impacting detection stability. In addition, decision-making strategies based on single-frame images are extremely sensitive to occasional image noise and motion artifacts, lacking the ability to verify detection over time.
[0003] In view of this, it is indeed necessary to improve the existing methods for detecting foreign objects in needles in order to solve the above problems. Summary of the Invention
[0004] To address the aforementioned technical problems, this invention provides a needle foreign object detection method. This method effectively suppresses interference from illumination fluctuations and mechanical jitter by using static background modeling and multi-frame fusion. Furthermore, the multi-frame judgment strategy based on trajectory lifetime threshold significantly reduces false alarms caused by single-frame noise and artifacts.
[0005] The technical solution of this invention is:
[0006] A method for detecting foreign objects in a needle includes the following steps:
[0007] Obtain the grayscale video sequence of the syringe and normalize each frame into a grayscale image of a uniform format;
[0008] Select one frame from the grayscale video sequence to perform static background modeling and generate a static background image;
[0009] Within a preset ROI or dynamic ROI region, calculate the difference image between the current frame and the static background image, and binarize the difference image to obtain a binary mask.
[0010] Extract the contour of the target in the binary mask, calculate the detection center point of the contour, associate the detection center points of each frame and construct the motion trajectory;
[0011] The lifespan of the motion trajectory is calculated, and the presence of foreign objects is determined based on whether the lifespan of the motion trajectory exceeds a preset threshold. The determination result is then output.
[0012] As a further improvement of the present invention, the construction of the dynamic ROI includes: generating a single-frame ROI based on the grayscale histogram of the current frame, and fusing the single-frame ROIs of multiple frames to obtain a dynamic ROI.
[0013] As a further improvement of the present invention, the construction of the dynamic ROI further includes: fusing single-frame ROIs from multiple frames based on a voting method to obtain a stable dynamic ROI, wherein the voting method includes:
[0014] Initialize a vote accumulator matrix with the same size as the image, with all elements initialized to zero;
[0015] For N consecutively acquired images, the corresponding single-frame ROIs are accumulated one by one into the voting accumulator matrix. The single-frame ROI is a binary image, where the pixel value belonging to the needle region is 1 and the pixel value not belonging to the needle region is 0.
[0016] Set a threshold α for the voting ratio;
[0017] Calculate the voting threshold T = α × N;
[0018] For each element in the voting accumulator matrix, if the element value is greater than or equal to T, then mark the corresponding pixel as belonging to the needle region in the stable ROI; otherwise, mark it as a non-needle region.
[0019] Morphological erosion was performed on the obtained stable ROI to obtain the stable ROI region.
[0020] As a further improvement of the present invention, selecting a frame from the grayscale video sequence for static background modeling includes: reordering the values of multiple grayscale frames based on each pixel, and selecting the frame with the second smallest value for static modeling to generate a static background image.
[0021] As a further improvement of the present invention, the reordering of the values of multiple grayscale frames for each pixel and the selection of the frame with the second smallest value includes:
[0022] The number of grayscale frames for each pixel is determined:
[0023] When the number of grayscale frames N≤8, the values of the multiple grayscale frames are reordered and the second smallest value is taken;
[0024] When the number of grayscale frames N>8, a strategy combining linear spatial rearrangement and quick sort is used to reorder the values of multiple grayscale frames and take the second smallest value.
[0025] As a further improvement of the present invention, extracting the contour of the candidate target in the binary mask includes: extracting the contour of the candidate target from the binary mask within a stable ROI, and filtering out contours that do not meet the preset geometric conditions.
[0026] As a further improvement of the present invention, within a stable ROI, the contours of candidate targets are extracted from the binary mask, and contours that do not meet preset geometric conditions are filtered out, specifically including:
[0027] Within the stable ROI region, the contours of candidate targets are extracted from the binary mask to obtain one or more candidate contours;
[0028] For each candidate contour, its geometric features are calculated and filtered according to preset geometric constraints, retaining only the contours that satisfy all constraints as valid defect candidates.
[0029] The geometric constraints include one or more of the following:
[0030] The outline area is greater than the preset minimum area threshold and less than the preset maximum area threshold.
[0031] The aspect ratio of the bounding rectangle is within a preset reasonable range;
[0032] The dimensions of the bounding rectangle of the outline meet the preset minimum width and minimum height requirements.
[0033] As a further improvement of the present invention, associating the detection center points of each frame with the motion trajectory includes: associating and matching the set of detection center points extracted in the current frame with the end points of one or more motion trajectories constructed based on historical frames; successfully matched points are used to extend the corresponding existing motion trajectory; unmatched points are used to initialize a new motion trajectory; wherein, the association matching is judged at least based on the spatial distance criterion and the motion direction consistency criterion.
[0034] As a further improvement of the present invention, associating the detection center point of each frame and constructing the motion trajectory further includes a trajectory termination step, which includes:
[0035] Regularly check the status of one or more established motion trajectories;
[0036] For each motion trajectory, determine whether it has failed to successfully match a new detection center point in the most recent N consecutive frames, where N is a preset positive integer;
[0037] If a trajectory meets the above conditions, then the trajectory is considered terminated.
[0038] The motion trajectory calculated in the lifetime calculation refers to the motion trajectory that still survives after the trajectory termination step.
[0039] As a further improvement of the present invention, the lifespan of the motion trajectory is calculated, and the presence of a foreign object is determined based on whether the lifespan of the motion trajectory exceeds a preset threshold. The determination result is then output, specifically including:
[0040] For each motion trajectory, calculate its lifetime L, where lifetime L is the total number of frames that the trajectory lasts from the starting frame to the current frame;
[0041] The lifespan L of each motion trajectory is compared with a preset lifespan threshold.
[0042] If the lifespan L of at least one motion trajectory is greater than or equal to the lifespan determination threshold, then it is determined that there is a foreign object, and the first determination result is output.
[0043] If the lifespan L of all motion trajectories is less than the lifespan determination threshold, it is determined that there is no foreign object, and a second determination result is output.
[0044] The beneficial technical effects of this invention are as follows: The needle foreign object detection method of this invention acquires a grayscale video sequence of the needle and standardizes each frame into a uniform grayscale image; selects one frame from the grayscale video sequence to perform static background modeling, generating a static background image; within a preset ROI or dynamic ROI region, calculates the difference image between the current frame and the static background image, and performs binarization processing on the difference image to obtain a binary mask; extracts the contour of the candidate target in the binary mask, calculates the detection center point of the contour, associates the detection center points of each frame, and constructs a motion trajectory; calculates the lifetime of the motion trajectory, and determines whether a foreign object exists based on whether the lifetime of the motion trajectory exceeds a preset threshold, and outputs the determination result. In this way, it can effectively suppress interference from illumination fluctuations and mechanical jitter, significantly reduce false alarms caused by single-frame noise and artifacts, and improve the detection effect of medical syringe production lines. Attached Figure Description
[0045] Figure 1 This is a flowchart of a needle foreign body detection method according to a preferred embodiment of the present invention. Detailed Implementation
[0046] In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0047] Please see Figure 1 As shown, this invention discloses a method for detecting foreign objects in a needle tube. The method includes:
[0048] Obtain the grayscale video sequence of the syringe and normalize each frame into a grayscale image of a uniform format;
[0049] Select one frame from the grayscale video sequence to perform static background modeling and generate a static background image;
[0050] Within a preset ROI or dynamic ROI region, calculate the difference image between the current frame and the static background image, and binarize the difference image to obtain a binary mask.
[0051] Extract the contour of the target in the binary mask, calculate the detection center point of the contour, associate the detection center points of each frame and construct the motion trajectory;
[0052] The lifespan of the motion trajectory is calculated, and the presence of foreign objects is determined based on whether the lifespan of the motion trajectory exceeds a preset threshold. The determination result is then output.
[0053] This effectively suppresses interference from light fluctuations and mechanical jitter, significantly reduces false alarms caused by single-frame noise and artifacts, and improves the detection effect of medical syringe production lines.
[0054] Specifically, acquiring the grayscale video sequence of the syringe and standardizing each frame into a uniform grayscale image mainly involves: receiving an 8-bit unsigned single-channel grayscale frame sequence of the same size and type {I1, I2, …, I n The frame verification function ensures the consistency of frame size and data type.
[0055] Selecting a frame from the grayscale video sequence for static background modeling and generating a static background image includes: reordering the values of multiple grayscale frames based on each pixel, selecting the frame with the second smallest value for static modeling, and generating a static background image.
[0056] The values of multiple grayscale frames for each pixel are reordered, and the frame with the second smallest value is selected, including:
[0057] The number of grayscale frames for each pixel is determined:
[0058] When the number of grayscale frames N≤8, the values of the multiple grayscale frames are reordered and the second smallest value is taken;
[0059] When the number of grayscale frames N > 8, a strategy combining linear spatial rearrangement and quicksort is used to reorder the values of multiple grayscale frames, and the second smallest value is taken. By using the second smallest value, extremely low-value interference caused by random noise in a single frame can be effectively avoided.
[0060] Specifically, an adaptive sorting and statistical strategy is used to generate a robust background model. For small sample cases where N ≤ 8 frames, a sorting operation is performed on each pixel position (x, y), and the second smallest value is taken as the background pixel value:
[0061]
[0062] Where sort[1] represents the second smallest element of the sorted sequence. For large samples N>8, linear space rearrangement and quicksort algorithms are used to optimize computational efficiency. Pixel standard deviation maps can be selectively calculated:
[0063]
[0064] Used for robust adjustment of dynamic thresholds.
[0065] Within a preset ROI or dynamic ROI region, calculate the difference image between the current frame and the static background image, and binarize the difference image to obtain a binary mask, which is represented as follows:
[0066] Calculate the absolute difference between the current frame and the background model:
[0067]
[0068] A dual threshold strategy is used for noise suppression. The binary mask generation rule is as follows:
[0069]
[0070] in Pixel-level threshold, The threshold for the regional mean. This represents the mean of the differences within the ROI. A morphological closing operation is then performed to smooth the void structure.
[0071] The construction of the dynamic ROI includes: generating a single-frame ROI based on the grayscale histogram of the current frame, and fusing the single-frame ROIs of multiple frames to obtain the dynamic ROI.
[0072] The construction of the dynamic ROI also includes: fusing single-frame ROIs from multiple frames based on a voting method to obtain a stable dynamic ROI, wherein the voting method includes:
[0073] Initialize a vote accumulator matrix with the same size as the image, with all elements initialized to zero;
[0074] For N consecutively acquired images, the corresponding single-frame ROIs are accumulated one by one into the voting accumulator matrix. The single-frame ROI is a binary image, where the pixel value belonging to the needle region is 1 and the pixel value not belonging to the needle region is 0.
[0075] Set a threshold α for the voting ratio;
[0076] Calculate the voting threshold T = α × N;
[0077] For each element in the voting accumulator matrix, if the element value is greater than or equal to T, then mark the corresponding pixel as belonging to the needle region in the stable ROI; otherwise, mark it as a non-needle region.
[0078] Morphological erosion was performed on the obtained stable ROI to obtain the stable ROI region.
[0079] Among them, the voting integration strategy includes: voting ratio Adaptive selection criteria:
[0080]
[0081] Specifically, the main peak value is determined based on the grayscale histogram analysis of the current frame:
[0082]
[0083] Constructing a bandpass mask If and only if Extracting the largest area connected region through connected component analysis:
[0084]
[0085] Calculate the geometric centroid of the connected domain as the flood fill seed point:
[0086]
[0087] The flood fill algorithm is executed starting from seed point S, with a tolerance set to ±10 grayscale values.
[0088]
[0089] Cross-frame voting fusion mechanism: cumulative voting on N-frame ROI results
[0090]
[0091] The final stable ROI is determined by the voting ratio threshold:
[0092]
[0093] Then perform the corrosion operation to shrink the boundary and obtain the final ROI.
[0094] In this embodiment, the difference and binarization need to be performed in a stable ROI. Therefore, dynamic fusion is required to obtain a stable ROI.
[0095] Difference and binarization include: calculating the absolute difference between the current frame and the background model, with the formula for the difference image D(x, y) as follows:
[0096]
[0097] A dual threshold strategy is used for noise suppression. The generation rule for the binary mask M(x,y) is as follows:
[0098]
[0099] in Pixel-level threshold, The threshold for the regional mean. This represents the mean of the differences within the ROI. A morphological closing operation is then performed to smooth the void structure.
[0100] Specifically, extracting the contours of candidate targets from the binary mask includes: within a stable ROI, extracting the contours of candidate targets from the binary mask and filtering out contours that do not meet preset geometric conditions.
[0101] Within a stable ROI, the contours of candidate targets are extracted from the binary mask, and contours that do not meet preset geometric conditions are filtered out. Specifically, this includes:
[0102] Within the stable ROI region, the contours of candidate targets are extracted from the binary mask to obtain one or more candidate contours;
[0103] For each candidate contour, its geometric features are calculated and filtered according to preset geometric constraints, retaining only the contours that satisfy all constraints as valid defect candidates.
[0104] The geometric constraints include one or more of the following:
[0105] The outline area is greater than the preset minimum area threshold and less than the preset maximum area threshold.
[0106] The aspect ratio of the bounding rectangle is within a preset reasonable range;
[0107] The dimensions of the bounding rectangle of the outline meet the preset minimum width and minimum height requirements.
[0108] Optionally, the contours of candidate masks are extracted within a stable ROI, and false targets are filtered out based on area and bounding box thresholds. The bounding box thresholds include a minimum area threshold (e.g., 2-10 pixels²) to filter out excessively small noise, a maximum area threshold (e.g., 20%-50% of the image area) to filter out excessively large background areas, and an aspect ratio threshold (e.g., 1:10 to 10:1) to filter out abnormal shapes that are too narrow or flat, thereby retaining candidate targets that conform to the actual defect characteristics.
[0109] Associating the detection center points of each frame with the motion trajectory involves: associating and matching the set of detection center points extracted in the current frame with the end points of one or more motion trajectories constructed based on historical frames; successfully matched points are used to extend the corresponding existing motion trajectory; unmatched points are used to initialize a new motion trajectory; wherein, the association matching is judged at least based on the spatial distance criterion and the motion direction consistency criterion.
[0110] Associating the detection center points of each frame with the motion trajectory also includes a trajectory termination step, which includes:
[0111] Regularly check the status of one or more established motion trajectories;
[0112] For each motion trajectory, determine whether it has failed to successfully match a new detection center point in the most recent M consecutive frames, where M is a preset positive integer;
[0113] If a trajectory meets the above conditions, then the trajectory is considered terminated.
[0114] Specifically, the motion trajectory calculated for the lifetime of the motion trajectory is the one that survives the trajectory termination step. This allows for further screening and elimination of motion trajectories, reducing the computational workload of lifetime calculation.
[0115] The lifespan of the motion trajectory is calculated, and the presence of a foreign object is determined based on whether the lifespan of the motion trajectory exceeds a preset threshold. The determination result is then output, specifically including:
[0116] For each motion trajectory, calculate its lifetime L, where lifetime L is the total number of frames that the trajectory lasts from the starting frame to the current frame;
[0117] The lifespan L of each motion trajectory is compared with a preset lifespan threshold.
[0118] If the lifespan L of at least one motion trajectory is greater than or equal to the lifespan determination threshold, then it is determined that there is a foreign object, and the first determination result (output is NG) is output.
[0119] If the lifespan L of all motion trajectories is less than the lifespan determination threshold, it is determined that there is no foreign object, and a second determination result (output is OK) is output.
[0120] Preferably, the lifespan threshold L is set as follows: when the target is a fast-moving defect, L is set to 2-3 frames to improve the response speed; when the target is a slow or static defect, L is set to 5-10 frames to ensure stability; when the environmental noise is high, L is set to 8-15 frames to suppress false detections; it is generally recommended that L=3 be used as the default value, which can be adjusted according to the frame rate, defect motion characteristics and noise level in the actual application scenario; it can output the visualization results of the superimposed detection box, trajectory and ROI and the JSON structured results.
[0121] Preferably, trajectory correlation and lifetime determination can be achieved by: constructing a set of detection center points. ,in Trajectory association is based on the Euclidean distance criterion:
[0122]
[0123] Combining the constraint of consistent motion direction:
[0124]
[0125] Track lifespan statistics: The determination rule is: if there exists a trajectory that satisfies... Output NG if the condition is not met, otherwise output OK.
[0126] Adaptive lifetime threshold: Adaptive calculation formula based on system parameters:
[0127]
[0128] Where fps is the video frame rate. This represents the typical number of frames the target device remains in. Recommended settings for different application scenarios: Fast motion defect detection: Slow / static defect detection: High-noise environment: .
[0129] In this embodiment, Boost.Asio is used to implement asynchronous TCP service for batch image reception and result return; cross-platform MAC+HMAC-SHA256 is used for license verification, and execution is performed after successful verification. This provides an asynchronous network interface and license management, making it easy to deploy.
[0130] In summary, the needle foreign object detection method of the present invention acquires a grayscale video sequence of the needle and standardizes each frame into a uniform grayscale image; selects one frame from the grayscale video sequence to perform static background modeling, generating a static background image; within a preset ROI or dynamic ROI region, calculates the difference image between the current frame and the static background image, and binarizes the difference image to obtain a binary mask; extracts the contour of the candidate target in the binary mask, calculates the detection center point of the contour, associates the detection center points of each frame, and constructs a motion trajectory; calculates the lifetime of the motion trajectory, and determines whether a foreign object exists based on whether the lifetime of the motion trajectory exceeds a preset threshold, and outputs the determination result. Specific effects include: anti-shake and anti-illuminance: the static background uses the second minimum value / sorting statistics to suppress occasional motion and noise. Dynamic region adaptation: the stable ROI of bandpass + flooding + multi-frame voting significantly reduces false detections. Trajectory-level robust determination: direction consistency + lifetime threshold suppresses single-frame artifacts and occasional noise. Real-time performance and engineering: Supports multi-threaded parallelism and SIMD vectorization, provides asynchronous network interfaces and authorization management, and is easy to deploy.
[0131] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A method for detecting foreign objects in a needle, characterized in that, Includes the following steps: Obtain the grayscale video sequence of the syringe and normalize each frame into a grayscale image of a uniform format; Select one frame from the grayscale video sequence to perform static background modeling and generate a static background image; Within a preset ROI or dynamic ROI region, calculate the difference image between the current frame and the static background image, and binarize the difference image to obtain a binary mask. Extract the contour of the target in the binary mask, calculate the detection center point of the contour, associate the detection center points of each frame and construct the motion trajectory; The lifespan of the motion trajectory is calculated, and the presence of foreign objects is determined based on whether the lifespan of the motion trajectory exceeds a preset threshold. The determination result is then output.
2. The needle foreign body detection method according to claim 1, characterized in that, The construction of the dynamic ROI includes: generating a single-frame ROI based on the grayscale histogram of the current frame, and fusing the single-frame ROIs of multiple frames to obtain the dynamic ROI.
3. The needle foreign body detection method according to claim 2, characterized in that, The construction of the dynamic ROI also includes: fusing single-frame ROIs from multiple frames based on a voting method to obtain a stable dynamic ROI, wherein the voting method includes: Initialize a vote accumulator matrix with the same size as the image, with all elements initialized to zero; For N consecutively acquired images, the corresponding single-frame ROIs are accumulated one by one into the voting accumulator matrix. The single-frame ROI is a binary image, where the pixel value belonging to the needle region is 1 and the pixel value not belonging to the needle region is 0. Set a threshold α for the voting ratio; Calculate the voting threshold T = α × N; For each element in the voting accumulator matrix, if the element value is greater than or equal to T, then mark the corresponding pixel as belonging to the needle region in the stable ROI; otherwise, mark it as a non-needle region. Morphological erosion was performed on the obtained stable ROI to obtain the stable ROI region.
4. The needle foreign body detection method according to claim 1, characterized in that, Selecting a frame from the grayscale video sequence for static background modeling includes: reordering the values of multiple grayscale frames based on each pixel, and selecting the frame with the second smallest value for static modeling to generate a static background image.
5. The needle foreign body detection method according to claim 4, characterized in that, The values of multiple grayscale frames for each pixel are reordered, and the frame with the second smallest value is selected, including: The number of grayscale frames for each pixel is determined: When the number of grayscale frames N≤8, the values of the multiple grayscale frames are reordered and the second smallest value is taken; When the number of grayscale frames N>8, a strategy combining linear spatial rearrangement and quick sort is used to reorder the values of multiple grayscale frames and take the second smallest value.
6. The needle foreign body detection method according to claim 3, characterized in that, Extracting the contours of candidate targets from the binary mask includes: within a stable ROI, extracting the contours of candidate targets from the binary mask and filtering out contours that do not meet preset geometric conditions.
7. The needle foreign body detection method according to claim 6, characterized in that, Within a stable ROI, the contours of candidate targets are extracted from the binary mask, and contours that do not meet preset geometric conditions are filtered out. Specifically, this includes: Within the stable ROI region, the contours of candidate targets are extracted from the binary mask to obtain one or more candidate contours; For each candidate contour, its geometric features are calculated and filtered according to preset geometric constraints, retaining only the contours that satisfy all constraints as valid defect candidates. The geometric constraints include one or more of the following: The outline area is greater than the preset minimum area threshold and less than the preset maximum area threshold. The aspect ratio of the bounding rectangle is within a preset reasonable range; The dimensions of the bounding rectangle of the outline meet the preset minimum width and minimum height requirements.
8. The needle foreign body detection method according to claim 1, characterized in that, Associating the detection center points of each frame with the motion trajectory involves: associating and matching the set of detection center points extracted in the current frame with the end points of one or more motion trajectories constructed based on historical frames; successfully matched points are used to extend the corresponding existing motion trajectory; unmatched points are used to initialize a new motion trajectory; wherein, the association matching is judged at least based on the spatial distance criterion and the motion direction consistency criterion.
9. The needle foreign body detection method according to claim 8, characterized in that, Associating the detection center points of each frame with the motion trajectory also includes a trajectory termination step, which includes: Regularly check the status of one or more established motion trajectories; For each motion trajectory, determine whether it has failed to successfully match a new detection center point in the most recent N consecutive frames, where N is a preset positive integer; If a trajectory meets the above conditions, then the trajectory is considered terminated. The motion trajectory calculated in the lifetime calculation refers to the motion trajectory that still survives after the trajectory termination step.
10. The needle foreign body detection method according to claim 9, characterized in that, The lifespan of the motion trajectory is calculated, and the presence of a foreign object is determined based on whether the lifespan of the motion trajectory exceeds a preset threshold. The determination result is then output, specifically including: For each motion trajectory, calculate its lifetime L, where lifetime L is the total number of frames that the trajectory lasts from the starting frame to the current frame; The lifespan L of each motion trajectory is compared with a preset lifespan threshold. If the lifespan L of at least one motion trajectory is greater than or equal to the lifespan determination threshold, then it is determined that there is a foreign object, and the first determination result is output. If the lifespan L of all motion trajectories is less than the lifespan determination threshold, it is determined that there is no foreign object, and a second determination result is output.