Image processing-based remote monitoring method for pharmaceutical production line

Abstract

The present application belongs to the technical field of image processing, and particularly relates to a medicine production line remote monitoring method based on image processing, which comprises the following steps: calculating the reflection intensity score according to the relative deviation of the gray value of the pixel points in the bubble cap region and the overall gray value of the region and the gradient amplitude, and then determining the gray validity weight; initializing the tablet center estimation, combining the gray validity weight with the angle between the gradient direction and the radial direction, calculating the edge quality score, and optimizing the tablet center position through iterative barycentric optimization; dividing the fan-shaped region with the optimized center as the origin, screening the reliable region and the region to be completed according to the final edge quality score, and determining the completion value by using the interpolation of the quality scores of the reliable regions on both sides; constructing the edge intensity map by comprehensively considering the gradient amplitude, the final edge score and the completion value, and performing tablet defect detection according to the Hough circle detection result of the edge intensity map. The present application eliminates the interference of reflection and tablet deviation, and improves the detection accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing. More particularly, the present application relates to a remote monitoring method for a medicine production line based on image processing. BACKGROUND

[0002] In the automatic production line of medicines, blister packaging is the mainstream packaging form, and its quality is directly related to the safety of medication and production efficiency. In order to ensure that each plate of medicine is not missing, multiple or mixed with irregular shaped tablets and other defects, an industrial vision detection system needs to perform real-time and accurate tablet recognition on the blister plate moving at high speed.

[0003] However, under the combined influence of blister material and production line lighting environment, the surface of the blister is prone to form mirror reflection at a specific illumination angle, resulting in irregular highlight patches, causing the tablet edge to appear broken, blurred in the reflection area, and even part of the tablet area is covered by high light, which cannot effectively extract the edge features. At the same time, during the packaging process, affected by factors such as feeding deviation and blister forming error, there is often a small positional shift in the blister cavity, and the geometric center does not coincide with the center of the blister cavity, which causes the detection algorithm based on a fixed area to be prone to positioning deviation, further increasing the difficulty of defect recognition.

[0004] The current mainstream technology relies on Hough circle transformation to recognize tablets. This method realizes positioning by detecting circular edges. However, Hough circle detection is highly sensitive to edge integrity. When the tablet edge is broken due to reflection, the algorithm is prone to miss detection. When the blister structure, reflection area or tablet shadow forms a pseudo-arc edge, the algorithm is prone to false detection. In addition, the traditional method usually assumes that the center of the tablet coincides with the center of the blister, or uses a global image processing strategy, which cannot adapt to the actual situation of tablet offset and local uneven illumination, resulting in unstable detection accuracy. SUMMARY

[0005] To solve the technical problems of unstable detection accuracy of tablet defects caused by reflection on the surface of the blister and tablet offset, and prone to miss detection and false detection, the present application provides a remote monitoring method for a medicine production line based on image processing, comprising:

[0006] For each blister area in the gray image of the blister card packaging plate of the medicine, the reflection intensity score of a pixel point is determined according to the relative deviation of the gray value of the pixel point from the overall gray of the blister area and the gradient amplitude of the pixel point, and the gray validity weight of the pixel point is determined based on the reflection intensity score; the center estimation of the medicine tablet is initialized, the edge quality score of the pixel point under the current center estimation is determined based on the gray validity weight of the pixel point, the included angle between the gradient direction and the radial direction of the pixel point under the current center estimation, the center position of the medicine tablet is optimized by iteratively calculating the barycenter of the edge quality score distribution, and the edge quality score after iteration convergence is taken as the final edge quality score; the blister area is evenly divided into a plurality of sector areas with the center of the medicine tablet as the origin, the reliable area and the area to be completed are screened based on the final edge quality score, and the completion value of the area to be completed is determined according to the final edge quality score of the pixel points in the reliable areas on both sides of the area to be completed; the edge strength of the pixel point is determined according to the gradient amplitude of the pixel point, the final edge score and the completion value of the area to be completed to which the pixel point belongs, and the edge strength is used to form an edge strength image; the edge strength image is subjected to Hough circle detection, and whether the blister area exists missing, overloading or irregular medicine tablets is judged according to the Hough circle detection result.

[0007] The present application accurately identifies and suppresses the light reflection interference area by comprehensively analyzing the gray distribution characteristics and gradient change characteristics of the pixel points in the blister area, solves the problem of highlight interference caused by mirror reflection on the surface of the aluminum foil, iteratively optimizes the center position of the medicine tablet, overcomes the center positioning deviation caused by the shift of the medicine tablet in the blister, and makes the subsequent analysis based on the real medicine tablet position instead of the blister center; the targeted edge completion strategy based on the edge quality evaluation only completes the edge in the limited area caused by light reflection, restores the broken edge of the medicine tablet, avoids misjudging the interference edge as the medicine tablet edge, and at the same time retains the original shape characteristics of the irregular medicine tablet; through the Hough circle detection of the edge strength image, the detection accuracy and anti-interference ability are significantly improved. The present application solves the problems of edge breakage, false edge generation and medicine tablet shift caused by light reflection interference in the traditional detection technology, and greatly improves the detection accuracy of missing, overloading and irregular medicine tablets.

[0008] Preferably, the reflection intensity score satisfies the expression: ; in the formula, represents the coordinates of any one pixel point in the blister area; represents the reflection intensity score of the pixel point ; represents the gray value of the pixel point ; represents the average gray of all pixel points in the blister area; represents the standard deviation of the gray values of all pixel points in the blister area; represents the gradient amplitude of the pixel point The gradient magnitude; This represents the maximum gradient amplitude within the blister area.

[0009] Based on the characteristics that the gray values ​​of pixels in the reflective area are significantly higher than those in the normal area, and that the edges of the reflective area are blurred and the gradient amplitude is low, this invention calculates the reflection intensity score by combining the deviation of the gray values ​​of pixels in the bubble area from the overall gray value of the area and the gradient amplitude characteristics of the pixels. This allows the reflection intensity score to accurately reflect the reflectivity of the pixels, providing a basis for the subsequent gray value validity weighting of pixels.

[0010] Preferably, determining the grayscale validity weight of a pixel based on its reflection intensity score includes: determining a reflection suppression factor for the pixel based on its reflection intensity score. In the formula, Represents the coordinates of any pixel within the bubble area; Represents pixels Reflective inhibitory factor; The threshold for reflection intensity score; Represents the linear rectified function. Represents the hyperbolic tangent function; obtains the grayscale validity weight of pixels based on the reflection suppression factor: , Represents pixels Gray-scale validity weight, This is the reflection suppression weighting coefficient.

[0011] Preferably, the method for obtaining the reflection intensity score threshold is as follows: calculating the average reflection intensity score of all pixels within the bubble area. and standard deviation ,Will As a threshold for reflection intensity score.

[0012] Preferably, the initialization of the tablet's center estimation includes: using the blister center position as the initialization of the tablet's center estimation.

[0013] Preferably, the edge quality score of the pixel under the current center estimation satisfies the expression: In the formula, Represents the coordinates of any pixel within the bubble area; Iteration Next time pixel Edge quality score, Represents pixels Gray-scale validity weight; Represents pixels gradient direction and pixel The included angle in the radial direction, where the pixel point the radial direction is the current center estimation to the pixel the direction pointed by the line; is a cosine function; is an absolute value symbol.

[0014] The present application considers that the gradient direction of the tablet edge pixel generally points to or departs from the tablet center, and thus determines the edge quality score of the pixel according to the cosine value of the included angle between the gradient direction of the pixel and the radial direction and the gray level effectiveness weight of the pixel, excludes the interference of the reflective region pixel, can distinguish the real tablet edge pixel and the non-edge pixel, makes the edge quality score accurately reflect the edge attribute of the pixel, and the new center estimation calculated based on the edge quality score of the pixel is offset to the real edge region of the edge quality score, gradually approaches the real center of the tablet, solves the direction deviation problem caused by the tablet offset, and provides a center positioning basis for subsequent edge completion and Hough circle detection.

[0015] Preferably, the filtering of the reliable region and the to-be-completed region based on the final edge quality score comprises: for each sector region, taking the pixel with the final edge quality score greater than a preset edge quality threshold in the sector region as a reliable edge point, and counting the proportion of the reliable edge point in all the pixels in the sector region; when the proportion is greater than a preset proportion threshold, marking the sector region as a reliable region; for any sector region of a non-reliable region, in response to the interval between the nearest reliable region on the left side of the sector region and the nearest reliable region on the right side being less than 10 sector regions, taking the sector region as a to-be-completed region.

[0016] Preferably, the determining of the completion value of the to-be-completed region comprises: performing weighted summation on the average value of the final edge quality scores of all the pixels in the nearest reliable region on the left side of the to-be-completed region and the average value of the final edge quality scores of all the pixels in the nearest reliable region on the right side of the to-be-completed region, to obtain the completion value of the to-be-completed region, wherein the weight of the nearest reliable region on the left side of the to-be-completed region is positively correlated with the distance from the to-be-completed region to the nearest reliable region on the right side, and the weight of the nearest reliable region on the right side of the to-be-completed region is positively correlated with the distance from the to-be-completed region to the nearest reliable region on the left side.

[0017] The application can make the obtained completion value conform to the transition law of the reliable edges on both sides, avoid the completion deviation caused by simple average, effectively connect the broken edges of the tablets caused by interference such as reflection, ensure the continuity and authenticity of the edges, and avoid introducing pseudo edges that do not conform to the trend of the real edges, so that the subsequent Hough circle detection can more accurately identify the tablet profile and reduce the missed detection and misjudgment problems caused by edge breakage.

[0018] Preferably, the edge strength satisfies the expression:

[0019] ;

[0020] In the formula, represents the coordinates of any one pixel point in the blister area; represents the edge strength of the pixel point ; represents the final edge quality score of the pixel point ; represents the maximum value in the final edge quality scores of all pixel points in the blister area; represents the gradient amplitude of the pixel point ; represents the completion value of the pixel point belonging to the region to be completed; represents a step function; represents the pixel point set of all completion regions.

[0021] For the pixel points in the non-completion region, the gradient amplitude, the final edge quality score and the edge strength of the pixel points are combined to highlight the real tablet edge and weaken the influence of non-edge pixels; for the pixel points in the completion region, the completion value is additionally added to conform to the trend of the real edge and effectively connect the broken edges caused by interference, so that the real tablet edge is more clearly highlighted in the edge strength map, while pseudo edges are avoided, high-quality image input is provided for subsequent Hough circle detection, missed detection and misjudgment caused by unclear or broken edge features are reduced, and the accuracy of judging missing, multiple and irregular tablets is improved.

[0022] Preferably, the method for judging whether the blister area has missing, multiple or irregular tablets according to the Hough circle detection result comprises: for each blister area, if the number of detected circles is 0, it is determined as missing or irregular tablets; if the number of detected circles is greater than 1, it is determined as multiple; if the number of detected circles is equal to 1, it is determined that the tablet exists, at this time, the difference between the radius of the detected circle and the standard radius of the tablet is calculated, and in response to the difference being greater than 3 pixels, it is determined as an irregular tablet.

[0023] The present application has the beneficial effects that: the present application accurately identifies and suppresses the light reflection interference area by comprehensively analyzing the gray scale distribution characteristics and gradient change characteristics of the pixel points in the blister area, solves the high light interference problem caused by the mirror reflection of the aluminum foil surface; through iterative optimization of the tablet center position, the center positioning deviation caused by the offset of the tablet in the blister is overcome, so that the subsequent analysis is based on the real tablet position instead of the blister center; the targeted edge completion strategy based on edge quality evaluation only completes the limited completion in the edge missing area caused by light reflection, which not only restores the broken tablet edge, but also avoids misjudging the interference edge as the tablet edge, while retaining the original shape characteristics of the special-shaped tablet; through Hough circle detection of the edge intensity graph, the detection accuracy and anti-interference ability are significantly improved. The present application solves the problems of edge breakage, false edge generation and tablet offset caused by light reflection interference in traditional detection technology, and greatly improves the detection accuracy of missed, multiple and special-shaped tablets. BRIEF DESCRIPTION OF DRAWINGS

[0024] Figure 1 is a flow chart illustrating the image processing-based remote monitoring method of the pharmaceutical production line in the present application;

[0025] Figure 2 is a schematic diagram of the original RGB image of the blister packaging plate of the medicine;

[0026] Figure 3 is a schematic diagram of the gray scale image of the blister packaging plate of the medicine;

[0027] Figure 4 is a schematic diagram of the standard mask;

[0028] Figure 5 is a schematic diagram of the blister area image;

[0029] Figure 6 is a schematic diagram of the image after contrast-limited adaptive histogram equalization of the blister area;

[0030] Figure 7 is a schematic diagram of the gradient image of the blister area;

[0031] Figure 8 is a schematic diagram of the fan-shaped area division schematic diagram;

[0032] Figure 9 is a schematic diagram of the edge intensity graph of the blister area;

[0033] Figure 10 is a schematic diagram of the result of Hough circle detection of the edge intensity graph of the blister area;

[0034] Figure 11 is a schematic diagram of the result of Hough circle detection of the gradient image of the blister area. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Since transparent blister packs are the mainstream form of pharmaceutical blister packaging, and round tablets are a common form of tablet-type pharmaceuticals, this invention focuses on testing transparent blister packs and round tablets. The specific embodiments of this invention are described in detail below with reference to the accompanying drawings.

[0037] This invention discloses a remote monitoring method for pharmaceutical production lines based on image processing, referring to... Figure 1 This includes steps S1 to S6:

[0038] S1: Collect grayscale images of blister packs for pharmaceutical products.

[0039] Specifically, an industrial CCD camera is used to capture images of blister packs on the pharmaceutical production line from a vertical, top-down angle, obtaining the original RGB image. The original RGB image is then converted to grayscale to obtain a grayscale image of the blister pack. For example, Figure 2 The original RGB image, Figure 3 This is a grayscale image of a pharmaceutical blister pack.

[0040] Optionally, Gaussian filtering can be applied to the grayscale image for noise reduction. It should be noted that Gaussian filtering of the grayscale image can suppress some ambient light interference and random noise while preserving key edge details between the pill and the background. Other noise reduction algorithms can also be used.

[0041] S2: Obtain the bubble region in the grayscale image and perform image enhancement on the bubble region.

[0042] It should be noted that pharmaceutical blister packaging has standard specifications, and the blister area is fixed in position and regular in shape on the production line. Therefore, this invention obtains each blister area in the grayscale image of the pharmaceutical blister packaging plate through a standard mask of the blister area.

[0043] Specifically, a standard mask is read, in which each blister region is marked as an independent connected component. The standard mask is multiplied by the grayscale image of the pharmaceutical blister packaging plate to obtain a mask extraction image. The region corresponding to each connected component in the standard mask in the mask extraction image is a blister region. For example, Figure 4 For standard masks, Figure 5 Image of the blister pack area.

[0044] Optionally, image enhancement is performed on each blister area respectively. In the present application, a contrast limited adaptive histogram equalization algorithm (CLAHE) is used to enhance each blister area, and other image enhancement algorithms can also be used by the implementer. Exemplarily, Figure 6 the image after contrast limited adaptive histogram equalization of the blister area.

[0045] It should be noted that the local illumination of the blister packaging plate of the medicine is quite different, and global histogram equalization of the gray image of the blister packaging plate of the medicine will destroy the original gray relationship and amplify the noise, while the present application independently enhances each blister area, which can improve the local contrast of the edge of the tablet in the blister and provide high-quality input for subsequent tablet detection.

[0046] S3: For each blister area, the reflection intensity score of the pixel point is determined according to the relative deviation of the gray value of the pixel point in the blister area from the overall gray of the blister area and the gradient amplitude of the pixel point, and the gray validity weight of the pixel point is determined based on the reflection intensity score.

[0047] It should be noted that due to the smooth surface of the blister, the blister area is prone to reflection, forming a reflective area, and the gray value of the reflective area is usually much higher than that of the surrounding area, forming a highlight spot. At the same time, the edge of the reflective area is usually blurred and the gradient changes gently, while the gray of the tablet inside the blister area is lower than that of the reflective area, and the edge of the tablet has obvious gradient change, so the present application confirms the reflection intensity score of the pixel point according to the gray value distribution characteristics and gradient change characteristics of the pixel point in the blister area.

[0048] Specifically, for each blister area, the gradient amplitude and gradient direction of each pixel point in the blister area are obtained by using a Sobel operator. Exemplarily, Figure 7 the gradient image of the blister area.

[0049] For each pixel point in the blister area, the reflection intensity score of the pixel point is determined according to the relative deviation of the gray value of the pixel point from the overall gray of the blister area and the gradient amplitude of the pixel point:

[0050] ;

[0051] In the formula, represents the coordinates of any one pixel point in the blister area; represents the reflection intensity score of the pixel point ; represents the gray value of the pixel point ; represents the average gray level of all the pixels in the blister region; represents the standard deviation of the gray level of all the pixels in the blister region; represents the gradient amplitude of pixel ; represents the maximum gradient amplitude in the blister region, used for normalizing ;

[0052] the reflection intensity score reflects the possibility of belonging to the highlight region, reflects the relative deviation of the gray level of pixel from the overall gray level of the blister region. Since the gray level of the pixel in the highlight region is larger than that of the surrounding region, when is larger, pixel is more likely to be in the highlight region. At the same time, the real highlight region usually has a fuzzy edge, so when is close to , tends to 0, thereby avoiding misjudging the clear edge as highlight.

[0053] Further, according to the reflection intensity score of the pixel in the blister region, the highlight suppression factor of the pixel is determined:

[0054] ;

[0055] wherein, represents the coordinates of any one pixel in the blister region; represents the highlight suppression factor of pixel ; is the reflection intensity score threshold; represents a linear rectifier function, in which when , , when , ; represents a hyperbolic tangent function, used to limit the highlight suppression factor of the pixel to the range of .

[0056] wherein, the reflection intensity score threshold is obtained by calculating the mean value and the standard deviation of the reflection intensity scores of all the pixels in the blister region, and .This serves as the threshold for reflection intensity scoring. It should be noted that this invention uses a range of 2 standard deviations because, according to statistical principles, 95% of normal data distributions fall within the range of mean ± 2 standard deviations. Since reflective areas constitute a relatively small proportion of the bubble area, the reflection intensity scores of pixels in reflective areas typically exceed... Therefore, this invention sets the reflection intensity score threshold. Set as This is used to distinguish between reflective and normal areas. Implementers can also adjust the standard deviation multiple based on the proportion of reflective areas within the blister pack area during actual implementation. For example... .

[0057] It should be noted that the reflection intensity score threshold reflects the statistical boundary of normal brightness variation within the bubble area, while the reflection suppression factor reflects the degree to which the reflection intensity score of a pixel exceeds the normal range. The more a pixel's reflection intensity score exceeds the reflection intensity score threshold, the more likely the pixel is to be in a reflective area, and the larger the reflection suppression factor is. When a pixel's reflection intensity score does not exceed the reflection intensity score threshold, the pixel is more likely to be in a normal area, and the reflection suppression factor is 0.

[0058] Furthermore, the grayscale validity weight of each pixel is obtained based on the reflection suppression factor:

[0059] ;

[0060] In the formula, Represents the coordinates of any pixel within the bubble area; Represents pixels Gray-scale validity weight; Represents pixels The reflection suppression factor. The grayscale effectiveness weight reflects the probability that a pixel is in a non-reflective area. When the reflection suppression factor is larger, the pixel is more likely to be in a reflective area. At this time, the grayscale effectiveness weight is smaller, which reduces the contribution of this area in subsequent processing and avoids the reflection area from interfering with the detection of the tablet. When the reflection suppression factor is 0, the grayscale effectiveness weight is 1, indicating that the pixel is not affected by reflection interference.

[0061] in, This is the reflection suppression weight coefficient, used to control the degree of influence of the reflection suppression factor on the grayscale effectiveness weight. It represents the system's sensitivity to reflection interference. The higher the value, the stronger the system's reflection suppression. The smaller the value, the less sensitive the system is to reflective interference, and the system exhibits weak reflective suppression. The value range of is set to (0,1]. In this invention, Set it to 0.6.

[0062] S4: Initialize the center estimation of the pill. Based on the gray-level validity weight of the pixel, the angle between the gradient direction and the radial direction of the pixel under the current center estimation, determine the edge quality score of the pixel under the current center estimation. Optimize the center position of the pill by iteratively calculating the centroid of the edge quality score distribution. The edge quality score after iterative convergence is taken as the final edge quality score.

[0063] It should be noted that tablets are usually offset within the blister pack, and their geometric center does not coincide with the blister center. Directly using the blister center for radial analysis will lead to directional deviation. Furthermore, the gradient direction of the tablet edge usually points towards or away from the center, either consistent with or opposite to the radial direction. Therefore, this invention estimates the tablet center position based on the angle between the gradient direction of the pixel and the radial direction, as well as the grayscale validity weight of the pixel.

[0064] Specifically, 1. For any blister region, initialize the center estimation of the tablet. 1. Determine the center coordinates of the blister pack; 2. Determine the edge quality score of each pixel under the current center estimate based on the angle between the gradient direction of the pixel and the radial direction of the pixel under the center estimate, as well as the gray-level validity weight of the pixel; 3. Calculate a new center estimate based on the edge quality scores of all pixels in the blister pack area under the current center estimate; 4. Repeat steps 2 to 3 until the convergence condition is met and the iteration stops. The edge quality score of each pixel under the latest center estimate at the time of iteration stop is taken as the final edge quality score of the pixel, and the latest center estimate is taken as the center of the pill.

[0065] In one embodiment, the number of iterations is denoted as The current central estimate is denoted as Determine the current central estimate Edge quality scores for each pixel:

[0066] ;

[0067] In the formula, Represents the coordinates of any pixel within the bubble area; Iteration Next time pixel Edge quality score, Represents pixels Gray-scale validity weight; Represents pixels gradient direction and pixel The included angle in the radial direction, where the pixel point The radial direction is the current center estimate. To pixel The direction to which the connection points. is a cosine function; is an absolute value symbol.

[0068] For the tablet region internal pixel points or background pixel points, the gradient direction is randomly distributed, and the angle between the pixel point radial direction and the gradient direction is uncertain; while the pixel points of the tablet edge point to the center or away from the center, which is consistent or opposite to the radial direction, so that tends to 0° or 180°, so that tends to 1, and the gray validity weight of the pixel points of the tablet edge is larger, so that the edge quality score of the pixel points is larger; for the reflective region pixel points, the gradient direction is random, and the gray validity weight is smaller, so that the edge quality score of the pixel points is smaller. Therefore, when the edge quality score is larger, the pixel point is more likely to be a tablet edge pixel point.

[0069] Further, according to the edge quality scores of all pixel points in the bubble cap region under the current center estimate , a new center estimate is calculated:

[0070] ;

[0071] ;

[0072] In the formula, represents the tablet center coordinate of the iteration ; represents the horizontal coordinate of the tablet center coordinate of the iteration ; represents the vertical coordinate of the tablet center coordinate of the iteration ; represents the edge quality score of the iteration pixel point under the tablet center estimate of the iteration ; represents the bubble cap region, represents the pixel point in the bubble cap region, represents the horizontal coordinate of the pixel point ; represents the vertical coordinate of the pixel point .

[0073] It should be noted that is the center of gravity of the high edge quality region, and when increases, the contribution of the point to the center estimate increases, so that the center is offset to the real tablet edge concentration area; when When the time is reduced, the contribution of the point pair center estimation is reduced. The present application updates the center estimation according to the distribution of the edge quality score of the pixel points in each iteration, so that the center estimation gradually approximates the true tablet center, thereby solving the problem of tablet deviation. When the tablet deviates to the right, the edge quality score of the pixel points on the right side is high, and the center estimation moves to the right. When the tablet deviates to the left, the edge quality score of the pixel points on the left side is high, and the center estimation moves to the left, so that the subsequent radial analysis is based on the true tablet position rather than the blister center.

[0074] It should be particularly noted that when the edge quality score of all pixel points in the blister area is 0, the distance between the center estimation and the true center is 0, the denominator in the expression of the center estimation will be zero, resulting in the inability to calculate the center estimation. This situation usually occurs when the blister area is completely covered by the reflective cover, the tablet is completely missing, or the image quality is extremely poor, resulting in the inability to extract any effective edge. Therefore, before each iteration calculation, the present application first checks whether is 0. If the value is 0, the iteration process is immediately terminated, and the blister area is marked as an edge feature missing state, and the subsequent steps are not performed. It is directly determined that the blister area has a serious defect, and manual review or rejection processing is prompted.

[0075] Further, a convergence condition is set: when the distance between and is less than a preset distance threshold or the maximum number of iterations is reached, the iteration is stopped.

[0076] It should be noted that the iteration process converges quickly, usually 2-3 iterations can obtain stable results, therefore the present application sets the maximum number of iterations to 3. In other embodiments, the implementer can set the maximum number of iterations according to the actual implementation. The distance threshold is used to control the convergence accuracy of the center estimation, which represents the acceptable range of center position changes. When the distance threshold is set larger, the iteration converges faster but the accuracy is lower. When the distance threshold is set smaller, the iteration converges slower but the accuracy is higher. In the present application, the distance threshold is set to 3 pixel distances. In other embodiments, the implementer can set the distance threshold according to the actual implementation. However, in order to ensure the accuracy of the center estimation, the distance threshold cannot exceed 5 pixel distances.

[0077] S5: Dividing the blister area into multiple sector regions with the tablet center as the origin, screening reliable regions and incomplete regions based on the final edge quality score, determining the completion value of the incomplete region according to the final edge quality score of the pixel points in the reliable regions on both sides of the incomplete region; determining the edge strength of the pixel points according to the gradient amplitude of the pixel points, the final edge score and the completion value of the incomplete region to which the pixel points belong, and constructing an edge strength map with the edge strength.

[0078] It should be noted that due to the reflection interference, the edges of the tablets may be broken or missing in a certain direction, resulting in false detection of the Hough circle detection, and therefore the present application combines the edge quality score of the pixel points and the angle continuity to complete the high-reliability area within a limited range, thereby improving the accuracy of the subsequent Hough circle detection.

[0079] Specifically, for each blister area, the blister area is divided into 72 fan-shaped areas with the center of the tablet as the center point, and the angle of each fan-shaped area is . Exemplarily, Figure 8 a schematic diagram for dividing the fan-shaped areas is shown.

[0080] Further, for each fan-shaped area, the pixel points in the fan-shaped area with the final edge quality score greater than the preset edge quality threshold value are regarded as reliable edge points, the proportion of the reliable edge points in the fan-shaped area to all the pixel points in the fan-shaped area is counted, and when the proportion is greater than a preset proportion threshold value, the fan-shaped area is marked as a reliable area.

[0081] It should be noted that the reliable area represents that the integrity of the edges of the tablet in the angle direction is higher. The edge quality threshold value is used to control the screening standard of the reliable edge points, and represents the requirement for the edge quality. When the edge quality threshold value is set to be larger, the reliable edge points screened out are fewer, and the requirement for the edge integrity is higher; when the edge quality threshold value is set to be smaller, the reliable edge points screened out are more, and the requirement for the edge integrity is lower. In the present application, the edge quality threshold value is set to be 0.4 , , which represents the maximum value in the final edge quality score of all the pixel points in the blister area. In other embodiments, the implementer can set the edge quality threshold value according to the actual implementation, but in order to ensure the accuracy of the edge completion, the edge quality threshold value should be no less than 0.2 and no more than 0.6 . The proportion threshold value is used to control the screening standard of the reliable area. When the proportion threshold value is set to be larger, the proportion of the reliable edge points in the fan-shaped area is required to be higher, and the screening of the reliable area is more strict; when the proportion threshold value is set to be smaller, the screening of the reliable area is more relaxed. In the present application, the proportion threshold value is set to be 0.3. In other embodiments, the implementer can set the proportion threshold value according to the actual implementation, but in order to ensure the reliability of the edge completion, the proportion threshold value should be no less than 0.2.

[0082] It should be noted that the proportion of the reliable edge points in the fan-shaped area to all the pixel points in the fan-shaped area reflects the edge integrity in the angle direction corresponding to each fan-shaped area.

[0083] Further, for any one sector of the non-reliable region, in response to the interval between the nearest reliable region on the left side of the sector and the nearest reliable region on the right side of the sector being less than 10 sectors, the sector is regarded as a region to be completed.

[0084] For any one region to be completed, according to the average of the final edge quality scores of all pixel points in the nearest reliable region on the left side of the region to be completed and the average of the final edge quality scores of all pixel points in the nearest reliable region on the right side of the region to be completed, the completion value of the region to be completed is calculated:

[0085] ;

[0086] In the formula, the completion value of the region to be completed is represented; the average of the final edge quality scores of all pixel points in the nearest reliable region on the left side of the region to be completed is represented; the average of the final edge quality scores of all pixel points in the nearest reliable region on the right side of the region to be completed is represented; the distance from the region to be completed to the nearest reliable region on the right side thereof is represented; the distance from the region to be completed to the nearest reliable region on the left side thereof is represented, and in the present application, the distance between the region to be completed and a reliable region is the number of sectors between the region to be completed and the reliable region plus 1, for example, if there are 3 sectors between a region to be completed and a reliable region, the distance from the region to be completed to the reliable region is 3+1=4.

[0087] The completion value is an interpolation based on the edge quality scores of the nearest reliable regions on both sides of the region to be completed, and reflects the real edge trend, when is smaller, the completion value is closer to the edge quality of the reliable region on the left side, and when is smaller, the completion value is closer to the edge quality of the reliable region on the right side, thereby ensuring that the completion value is consistent with the real edge trend.

[0088] It should be noted that the present application strictly limits the completion range, and only when the non-reliable region has reliable edges on both sides and the interval is small, the non-reliable region is regarded as a region to be completed for completion, which can prevent the interference edge from being mistaken for a tablet edge for completion, and when the interval between the nearest reliable region on the left side of the non-reliable region and the nearest reliable region on the right side is greater than or equal to 10, the edge of the non-reliable region is kept broken to avoid introducing a pseudo edge.

[0089] Furthermore, for each pixel within the blister region, in response to a pixel not belonging to the region to be completed, the edge intensity of the pixel is determined based on the pixel's gradient magnitude and final edge quality score. In response to a pixel belonging to the region to be completed, the edge intensity of the pixel is determined based on the pixel's gradient magnitude, final edge score, and the completion value of the region to be completed to which the pixel belongs. Specifically:

[0090] ;

[0091] In the formula, Represents the coordinates of any pixel within the bubble area; Represents pixels Edge strength; Represents pixels The final edge quality score; This represents the maximum value among all pixels within the bubble area in the final edge quality score; Represents pixels The gradient magnitude; This represents the padding value of the region to be padded to the pixel. Represents the step function, when hour, Preserve the original final edge quality score of each pixel. hour, ; This represents the set of pixels in the complete region.

[0092] When a pixel does not belong to the region to be filled, if the gradient magnitude of the pixel is larger, the final edge quality score of the pixel is also larger, and the final edge quality score is greater than... When a pixel is more likely to be a real pill edge, its edge strength is greater; when the final edge quality score of a pixel is smaller, and less than or equal to... When the pixel is less likely to be the edge of a real pill, the edge intensity of the pixel is smaller.

[0093] When a pixel belongs to the region to be completed, its edge strength is obtained based on its final edge quality score and the completed value. The larger the pixel's gradient magnitude, the higher its final edge quality score, or the larger the completed value of its region, the more likely the pixel is to be a real pill edge, and thus the stronger its edge strength. Conversely, the smaller the pixel's gradient magnitude, the lower its final edge quality score, and the smaller the completed value of its region, the less likely the pixel is to be a real pill edge, and thus the weaker its edge strength.

[0094] Furthermore, the edge intensity of all pixels within the blister region is used to construct an edge intensity map of the blister region. For example, Figure 9 This is the edge intensity map of the blister region.

[0095] S6. Perform Hough circle detection on the edge intensity map. Based on the Hough circle detection results, determine whether there are missing, overfilled, or irregularly shaped tablets in the blister pack area.

[0096] Specifically, Hough circle detection is performed on the edge intensity map of the blister region. During the Hough circle detection process, the search range for the center of the circle is set to be centered on the tablet. Centered on A circular region with a defined boundary, wherein, The radius of the blister area.

[0097] For each blister pack area, check the number of detected circles. If the number is 0, it is determined to be a missing or irregularly shaped tablet; if the number is greater than 1, it is determined to be an overfill; if the number is equal to 1, it is determined to be a tablet. At this time, calculate the difference between the radius of the detected circle and the standard radius of the tablet. If the difference is greater than 3 pixels, it is determined to be an irregularly shaped tablet.

[0098] For example, Figure 10 The results of Hough circle detection are shown for the edge intensity map of the blister region. Figure 11 To directly Figure 7 The results of Hough circle detection on the gradient image of the blister pack region shown are as follows: Directly performing Hough circle detection on the gradient image is affected by reflective areas and missing edges. In some blister pack regions, tablets are present but no circle is detected; in others, a circle is detected, but it does not coincide with the edge, resulting in offset. However, using the method of this invention to perform Hough circle detection on the edge intensity map can detect the position of all tablets, and the detected circle basically coincides with the edge of the tablet, effectively avoiding missed and false detections.

[0099] This enables the detection of missing, overfilled, and irregularly shaped tablets in blister packaging for pharmaceuticals.

Claims

1. A method for remote monitoring of a pharmaceutical production line based on image processing, characterized in that, The method comprises the following steps: For each blister area in the gray scale image of the drug blister packaging plate, the reflection intensity score of a pixel point is determined according to the relative deviation of the gray scale value of the pixel point in the blister area from the overall gray scale of the blister area and the gradient amplitude of the pixel point, comprising: ; wherein, represents the coordinates of any one pixel point in the blister area; represents the reflection intensity score of the pixel point ; represents the gray scale value of the pixel point ; represents the average gray scale of all pixel points in the blister area; represents the standard deviation of the gray scale values of all pixel points in the blister area; represents the gradient amplitude of the pixel point ; represents the maximum gradient amplitude in the blister area, the gray scale validity weight of the pixel point is determined based on the reflection intensity score, comprising: determining the reflection light suppression factor of the pixel point according to the reflection intensity score: , wherein, represents the reflection light suppression factor of the pixel point ; is a reflection intensity score threshold value; represents a linear rectification function, represents a hyperbolic tangent function; the gray scale validity weight of the pixel point is obtained according to the reflection light suppression factor: , represents the gray scale validity weight of the pixel point , is a reflection light suppression weight coefficient; Initializing a center estimation of the tablet, determining an edge quality score of a pixel under the current center estimation based on a gray scale validity weight of the pixel, an angle between a gradient direction of the pixel and a radial direction of the pixel under the current center estimation, comprising: ; wherein, represents an iteration times, an edge quality score of a pixel , represents an angle between a gradient direction of a pixel and a radial direction of a pixel , wherein the radial direction of the pixel is a direction pointed by a line connecting the current center estimation to the pixel ; is a cosine function; is an absolute value symbol, optimizing the tablet center position by iteratively calculating a barycenter of the edge quality score distribution, and taking the edge quality score after the iteration converges as a final edge quality score; The blister area is evenly divided into multiple sector areas with the tablet center as the origin, reliable areas and incomplete areas are screened based on the final edge quality score, the completion value of the incomplete area is determined according to the final edge quality score of the pixel points in the reliable areas on both sides of the incomplete area, comprising: ; wherein, represents the completion value of the incomplete area; represents the average value of the final edge quality scores of all pixel points in the nearest reliable area on the left side of the incomplete area; represents the average value of the final edge quality scores of all pixel points in the nearest reliable area on the right side of the incomplete area; represents the distance from the incomplete area to the nearest reliable area on the right side; represents the distance from the incomplete area to the nearest reliable area on the left side; the edge strength of the pixel point is determined according to the gradient amplitude of the pixel point, the final edge score and the completion value of the incomplete area to which the pixel point belongs, comprising: ; wherein, represents the edge strength of the pixel point ; wherein, represents the final edge quality score of the pixel point ; wherein, represents the maximum value in the final edge quality scores of all pixel points in the blister area; represents the completion value of the incomplete area to which the pixel point belongs; represents the step function; represents the pixel point set of all completed areas, and the edge strength is constructed into an edge strength map. The method comprises the following steps:

2. The image processing-based remote monitoring method for pharmaceutical production lines according to claim 1, characterized in that, The initialization of the center estimation of the tablet comprises the following steps: Calculate the mean of the reflection intensity scores of all pixels within the bubble area. and standard deviation ,Will As a threshold for reflection intensity score.

3. The image processing-based remote monitoring method for pharmaceutical production lines according to claim 1, characterized in that, The center position of the blister is taken as the initialization of the center estimation of the tablet. The method comprises the following steps:

4. The image processing-based remote monitoring method for pharmaceutical production lines according to claim 1, characterized in that, For each sector region, the pixel points with the final edge quality score greater than the preset edge quality threshold in the sector region are taken as reliable edge points, and the proportion of the reliable edge points in all pixel points in the sector region is counted; when the proportion is greater than a preset proportion threshold, the sector region is marked as a reliable region; for any sector region of a non-reliable region, if the interval between the nearest reliable region on the left side of the sector region and the nearest reliable region on the right side is less than 10 sector regions, the sector region is taken as a region to be completed. The method comprises the following steps:

5. The image processing-based remote monitoring method for pharmaceutical production lines according to claim 1, characterized in that, The average value of the final edge quality scores of all pixel points in the nearest reliable region on the left side of the region to be completed and the average value of the final edge quality scores of all pixel points in the nearest reliable region on the right side of the region to be completed are weighted and summed to obtain the completion value of the region to be completed, wherein the weight of the nearest reliable region on the left side of the region to be completed is positively correlated with the distance from the region to be completed to the nearest reliable region on the right side, and the weight of the nearest reliable region on the right side of the region to be completed is positively correlated with the distance from the region to be completed to the nearest reliable region on the left side. The method comprises the following steps:

6. The image processing-based remote monitoring method of a pharmaceutical production line according to claim 1, characterized by, For each blister region, if the number of detected circles is 0, it is determined to be a missing or abnormal-shaped tablet; if the number of detected circles is greater than 1, it is determined to be a multiple loading; if the number of detected circles is equal to 1, it is determined that the tablet exists, and at this time, the difference between the radius of the detected circle and the standard radius of the tablet is calculated; if the difference is greater than 3 pixels, it is determined to be an abnormal-shaped tablet. ​

Images