Medicine packaging defect detection system based on machine vision

Abstract

The invention provides a medicine package defect detection system based on machine vision, relates to the technical field of defect detection, and realizes accurate reduction of the surface morphology of a medicine package through a high-precision three-dimensional imaging technology based on structured light coding in combination with multi-angle parallax reconstruction and multi-channel disturbance analysis. And on the basis, a depth gradient difference graph and a disturbance graph are constructed to effectively capture tiny abnormal characteristics such as sunken defects and structural disturbance, so that the sensing capability of the system on fine-grained defects is improved. By constructing a redundant block analysis mechanism, when a two-dimensional code area is shielded or damaged, content restoration and logic verification can still be performed by using a cross-frame redundant structure, so that the risk of misjudgment caused by decoding failure of the two-dimensional code is effectively reduced, and the robustness of the system in various interference scenes is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and more particularly, to a drug packaging defect detection system based on machine vision. Background Art

[0002] With the continuous improvement of the level of intelligent manufacturing in the pharmaceutical industry, automated inspection technology for drug packaging quality has become an increasingly important part of ensuring drug safety and compliant production. Traditional manual visual inspection methods have been unable to meet the requirements of modern production for high-speed, high-precision and high-consistency inspection due to problems such as strong subjectivity, low efficiency and high fatigue. To this end, machine vision technology has gradually been applied to drug packaging defect detection, using industrial cameras to acquire images and combine them with image processing algorithms to identify and classify packaging surface defects. In recent years, technologies such as structured light 3D reconstruction, multimodal image fusion, and deep learning recognition have continued to mature, promoting the transformation of defect detection systems from static analysis of two-dimensional images to three-dimensional surface information reconstruction and spatial correlation analysis. The detection dimension and accuracy have been significantly improved, and it can handle more complex appearance defects and finer-grained structural variations.

[0003] However, there are still a series of technical limitations in the existing pharmaceutical packaging defect detection system, especially in the extraction of multi-angle structural information, defect labeling under complex texture interference, and redundant coding fault-tolerant recognition in areas with dense packaging information (such as QR code areas). Problems such as low detection accuracy, high misjudgment rate, and inaccurate edge positioning are still faced. On the one hand, existing systems mostly use single-frame images for defect analysis, which makes it difficult to effectively reconstruct the true three-dimensional structure of the pharmaceutical packaging surface. They are easily disturbed by factors such as uneven lighting and surface reflections, resulting in the inability to accurately identify tiny depressions or protrusions. On the other hand, in functional structural interference areas such as QR code areas, due to the certain fault tolerance and redundancy of the coding structure itself, conventional defect judgment models have difficulty distinguishing the boundary between packaging damage and fault-tolerant design. In addition, defect labeling often relies on manually preset rules or static thresholds, lacking a dynamic credibility fusion mechanism, making it difficult to achieve highly reliable judgment and final confirmation of suspected defect areas, resulting in some defects being missed or misjudged, affecting the stability and consistency of packaging quality control.

[0004] Therefore, a drug packaging defect detection solution based on machine vision is needed. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention is proposed. The present invention provides a drug packaging defect detection system based on machine vision.

[0006] According to one aspect of the present invention, a machine vision-based pharmaceutical packaging defect detection system is provided, comprising: a structured light encoding acquisition module, which performs multi-angle encoding projection on the pharmaceutical packaging surface using Gray code structured light stripes, uses a high-resolution camera to capture a sequence of stripe deformation images, and reconstructs a spatial coordinate depth map based on multi-frame projection differences; a defect annotation module, which performs depth gradient difference calculation on the spatial coordinate depth map and constructs a disturbance map to identify initial abnormal areas; extracts suspicious concave areas based on preset thresholds in the disturbance map, performs primary defect annotation based on contour boundary continuity, and generates a preliminary defect labeling map; a redundant block parsing module, which performs structural block segmentation on the two-dimensional code area in the pharmaceutical packaging image, extracts standard code block groups, and constructs a spatial index that is jointly matched with the preliminary defect labeling map; when the preliminary defect labeling map covers the two-dimensional code area and the initial decoding of the two-dimensional code fails, performs cross-frame displacement sliding window scanning and code block redundant structure comparison to generate a logical check matrix; and a defect credibility fusion module, which fuses the preliminary defect labeling map with the logical check matrix, generates a final defect confirmation map based on credibility parameters, and performs pharmaceutical packaging defect detection.

[0007] Furthermore, the multi-angle coded projection includes: using a multi-frequency Gray code structured light coding method to construct a coding sequence set composed of a high-frequency fine coding subsequence and a low-frequency coarse coding subsequence; projecting the coding sequence onto the surface of the drug packaging at multiple fixed angles in turn, and synchronously recording the corresponding projection image sequence at each angle, and at the same time establishing an angle association index matrix to mark the spatial correspondence.

[0008] Furthermore, the reconstruction of the spatial coordinate depth map based on multi-frame projection differences includes: performing phase difference processing between adjacent frames of the projection image sequence, constructing a multi-channel fringe disturbance map to describe the fringe offset distribution caused by surface morphology changes; using the multi-angle channels corresponding to the multi-channel fringe disturbance map to perform pixel-level joint fitting, reconstructing the joint disparity map, and extracting continuous morphology gradients through sub-pixel interpolation technology; converting the joint disparity map into a spatial coordinate depth map, and outputting it as a structured light depth map sequence.

[0009] Furthermore, the identification of initial abnormal areas includes: extracting spatial micro-perturbation gradient values ​​between adjacent frames from a structured light depth map sequence and constructing a perturbation map; mapping the perturbation map into a directional perturbation intensity map using a directional projection operator; constructing a fixed-size perturbation stability window centered on each pixel point (x, y), and counting the directional perturbation change rate within the window, and extracting an initial abnormal area set by setting a perturbation intensity threshold.

[0010] Furthermore, the generation of the preliminary defect marking map includes: performing contour tracking on the connected domain extracted from the initial abnormal area set to construct a multi-scale contour continuity factor; filtering and correcting the boundaries of the abnormal area according to the boundary closure and direction consistency at different scales, and outputting an optimized area set; using the corresponding positions of the optimized area set in the spatial coordinate depth map to construct a depression morphology feature map, extracting the average depression depth and disturbance sharpness in the area, combining with a preset depression judgment threshold, eliminating false alarm areas caused by packaging undulations or pattern interference, and outputting a defect confirmation candidate set; mapping all connected areas in the defect confirmation candidate set into binary mask images, and generating a preliminary defect marking map based on the area labels.

[0011] Furthermore, the generation of the logic check matrix includes: in a multi-frame image sequence, performing pixel-level sliding window scanning with a certain code block as the initial window, comparing the redundant structure difference between the sliding window area and the corresponding code block in the adjacent frame, and selecting the frame with the strongest redundant consistency to construct a code block check set; based on the structural consistency comparison results of each code block in the code block check set and the standard code block group, establishing a two-dimensional logic check matrix, in which the elements represent the reliability level of each code block under redundant structure verification.

[0012] Furthermore, the fusion process is to perform pixel-by-pixel logical weighted fusion on the preliminary defect marking image and the logic check matrix to form a joint response image.

[0013] Furthermore, the average response intensity of each connected area is calculated as a credibility indicator based on the joint response map, the values ​​of all pixel positions are expanded to the credibility map of the entire area, and all areas below the credibility threshold are eliminated to obtain the final defect confirmation map; if the coverage area of ​​the final defect confirmation map includes the QR code structure area, and the original QR code decoding fails, the area in the final defect confirmation map that overlaps with the QR code area is marked as a structural damage defect, prompting that structural repair is required; otherwise, the final defect confirmation map is output as the final defect detection result.

[0014] According to one aspect of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the machine vision-based pharmaceutical packaging defect detection system as described above.

[0015] According to one aspect of the present application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to implement the machine vision-based pharmaceutical packaging defect detection system provided in the various optional implementations described above.

[0016] Compared with the existing technology, the machine vision-based pharmaceutical packaging defect detection system provided by the present invention realizes the accurate restoration of the surface morphology of pharmaceutical packaging through high-precision three-dimensional imaging technology based on structured light coding, combined with multi-angle parallax reconstruction and multi-channel perturbation analysis, and constructs a depth gradient difference map and a perturbation map on this basis, effectively capturing small abnormal features such as concave defects and structural perturbations, thereby improving the system's perception of fine-grained defects. By constructing a redundant block parsing mechanism, when the QR code area is blocked or damaged, the cross-frame redundant structure can still be used for content restoration and logical verification, effectively reducing the risk of misjudgment caused by the failure of QR code decoding, and improving the robustness of the system in multiple interference scenarios. In addition, the present invention also has a credibility fusion mechanism, which dynamically generates a credibility response map by combining defect labeling results and redundant verification information, and eliminates low-credibility areas, thereby significantly improving the accuracy of defect confirmation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0018] Figure 1 4 is a system block diagram of a machine vision-based pharmaceutical packaging defect detection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0020] As mentioned in the above background technology, there are still a series of technical limitations in the existing pharmaceutical packaging defect detection system, especially in terms of multi-angle structural information extraction, defect labeling under complex texture interference, and redundant coding fault-tolerant recognition of packaging information-intensive areas (such as QR code areas). It still faces problems such as low detection accuracy, high misjudgment rate, and inaccurate edge positioning. On the one hand, the existing systems mostly use single-frame images for defect analysis, which makes it difficult to effectively reconstruct the true three-dimensional structure of the pharmaceutical packaging surface. It is easily disturbed by factors such as uneven lighting and surface reflection, resulting in the inability to accurately identify tiny concave or convex defects. On the other hand, in functional structural interference areas such as QR code areas, since the coding structure itself has certain fault tolerance and redundancy, conventional defect judgment models find it difficult to distinguish the boundary between packaging damage and fault-tolerant design. In addition, defect labeling often relies on manually preset rules or static thresholds, lacks a dynamic credibility fusion mechanism, and is difficult to achieve high-reliability judgment and final confirmation of suspicious defect areas, resulting in some defects being missed or misjudged, affecting the stability and consistency of packaging quality control. Therefore, there is a need for a pharmaceutical packaging defect detection solution based on machine vision.

[0021] Figure 1 FIG. 1 is a system block diagram of a machine vision-based drug packaging defect detection system according to an embodiment of the present invention. Figure 1 As shown in the figure, the machine vision-based pharmaceutical packaging defect detection system includes:

[0022] The structured light coding acquisition module 100 uses Gray code structured light stripes to perform multi-angle coding projection on the surface of the pharmaceutical packaging, and uses a high-resolution camera to capture a sequence of stripe deformation images, and reconstructs a spatial coordinate depth map based on the difference in multi-frame projections. The defect annotation module 200 performs depth gradient difference calculation on the spatial coordinate depth map and constructs a disturbance map to identify the initial abnormal area. Suspicious concave areas are extracted according to a preset threshold in the disturbance map, and primary defect annotation is performed based on the continuity of the contour boundary to generate a preliminary defect marking map. The redundant block parsing module 300 performs structural block segmentation on the QR code area in the pharmaceutical packaging image, extracts standard code block groups, and constructs a spatial index that is jointly matched with the preliminary defect marking map. When the preliminary defect marking map covers the QR code area and the QR code fails to be decoded for the first time, cross-frame displacement sliding window scanning and code block redundant structure comparison are performed to generate a logical check matrix. The defect credibility fusion module 400 fuses the preliminary defect marking map with the logical check matrix, generates a final defect confirmation map based on the credibility parameters, and performs pharmaceutical packaging defect detection.

[0023] In an embodiment of the present invention, the structured light encoding acquisition module 100 is used to perform high-precision three-dimensional reconstruction of the surface of pharmaceutical packaging. Its operating principle is based on the multi-angle encoding projection and high-resolution imaging acquisition technology of Gray code structured light. Specifically, by projecting structured light onto the pharmaceutical packaging surface at different spatial angles and simultaneously capturing a sequence of deformed fringe images, it achieves submillimeter reconstruction of the surface topography of the inspected object. This system is designed to provide a comprehensive, non-contact, high-precision scan of the packaging surface, making it particularly suitable for more three-dimensional and accurate defect identification and analysis in scenarios where small defects are not easily detected by conventional two-dimensional images.

[0024] The multi-angle coding projection includes: using a multi-frequency Gray code structured light coding method to construct a coding sequence set composed of a high-frequency fine coding subsequence and a low-frequency coarse coding subsequence; projecting the coding sequence onto the surface of the drug packaging at multiple fixed angles, and synchronously recording the corresponding projection image sequence at each angle, while establishing an angle association index matrix to mark the spatial correspondence.

[0025] It should be noted that in the structured light projection stage, in order to achieve high-precision, anti-interference image coding effects, the present invention adopts a multi-frequency Gray code coding strategy to design the projection stripes. Among them, Gray code is a coding method in which adjacent codes differ by only one bit, and has strong robustness and good decoding error tolerance. Its multi-frequency extension form refers to combining high-frequency detail Gray code subsequences with low-frequency contour Gray code subsequences to construct a coding sequence set, so as to effectively encode the packaging surface at different scale levels. The high-frequency subsequence can accurately capture the changes in surface microstructure and is suitable for identifying small defects such as slight indentations and fine wrinkles; while the low-frequency subsequence is suitable for coarse positioning of the overall surface morphology, thereby providing structural auxiliary information on a macro scale. This dual-scale coding structure can effectively suppress the coding ambiguity problem in the context of high reflection or complex patterns, and improve the subsequent reconstruction accuracy and robustness.

[0026] During the multi-angle projection process, the present invention employs multiple fixed-angle projection devices, each corresponding to a specific projection subchannel. During execution, the system sequentially projects a Gray code structured light sequence onto the pharmaceutical packaging surface from various angles, while simultaneously capturing a sequence of reflected fringe images at each angle using a high-resolution camera. Each angle corresponds to a set of time-sequentially projected images, and the images from all angles together form a complete multi-view imaging sequence. During this process, a set of angle-association indices is also constructed, aligning each projection angle with a frame index in the image sequence. This allows accurate identification of the projection angle corresponding to each fringe deformation image during subsequent processing.

[0027] The method for reconstructing a spatial coordinate depth map based on multi-frame projection differences includes: performing phase difference processing between adjacent frames of a projection image sequence to construct a multi-channel fringe disturbance map to describe the distribution of fringe offsets caused by surface topography changes; performing pixel-level joint fitting using multi-angle channels corresponding to the multi-channel fringe disturbance map to reconstruct a joint disparity map, and extracting continuous topography gradients through sub-pixel interpolation technology; and converting the joint disparity map into a spatial coordinate depth map, which is output as a sequence of structured light depth maps.

[0028] Specifically, during the image reconstruction phase, the module processes the captured multi-frame projection image sequence, first constructing a multi-channel fringe disturbance map through phase difference analysis between adjacent frames. This disturbance map is generated based on the deformation of the fringe on the object surface caused by differences in height or curvature at different projection angles. By calculating the phase changes of these deformations in different frames, the degree of fringe disturbance in the corresponding surface area is determined.

[0029] Before converting the joint disparity map into a spatial coordinate depth map, it is necessary to estimate the differential curvature of the spatial neighborhood of each pixel: based on the normal vector gradient and principal curvature direction of the 3D reconstructed surface, the Gaussian curvature and mean curvature of the local area are calculated to effectively classify surface arches (positive curvature) and depressions (negative curvature), thereby avoiding misjudgment caused by simply using depth drop as the basis for depression.

[0030] Specifically, the system uses the depth gradient variation within the surrounding neighborhood of each pixel's corresponding three-dimensional coordinate point to first extract the surface fitting curvature values ​​in the horizontal and vertical directions through principal direction slicing. It then infers Gaussian and mean curvatures based on the Laplace surface difference method. If an area has negative Gaussian curvature (such as a saddle surface), it is automatically marked as an atypical defect area and enters the next step of the context verification module. If the area has positive curvature and a certain depth threshold exists, it is preferentially marked as a "protrusion" rather than a "depression."

[0031] It is worth noting that traditional methods often only process images from a single angle, making it difficult to effectively identify phase detail differences caused by tiny bump changes. The perturbation map constructed by the present invention through multi-angle channels has significant advantages in descriptive ability and data redundancy, which helps to significantly improve the overall reconstruction accuracy.

[0032] After obtaining the multi-channel stripe perturbation map, the system further adopts a pixel-level joint fitting strategy for processing. The core idea of ​​this strategy is to jointly consider the perturbation values ​​of each pixel at various angles to form a joint disparity vector sequence. Subsequently, the sequence is calculated by the multi-channel joint fitting algorithm to obtain the optimal consistency matching position of the pixel at each angle, and then a high-precision joint disparity map is solved. It is worth emphasizing that in the joint fitting process, the present invention also introduces a sub-pixel interpolation algorithm at the sub-pixel level, which further refines the depth gradient changes of each pixel through gradient interpolation and regional boundary smoothing algorithms on the basis of integer pixels to capture more delicate surface contour information. This interpolation strategy can greatly improve the smoothness and continuity of the depth map while ensuring computational efficiency, avoiding the common sawtooth depth jump problem in conventional structured light systems.

[0033] Finally, the reconstructed joint disparity map is further converted into a spatial coordinate depth map. This conversion process, based on the projection-imaging space mapping relationship established during the system calibration phase, maps pixel disparity values ​​into three-dimensional spatial coordinate information, thereby forming a depth image with real-world geometric meaning. By integrating the depth maps obtained from multiple angles, a continuous sequence of structured light depth maps can be formed. This not only enables high-fidelity acquisition of the three-dimensional topography of the pharmaceutical packaging surface, but also effectively enhances the system's spatial recognition capabilities for complex defect structures (such as dents, warping, bubbles, etc.).

[0034] As can be seen, this invention combines multi-frequency Gray code projection, multi-angle synchronous imaging, and multi-channel joint reconstruction to form a high-precision, highly robust, and highly adaptable 3D acquisition system for pharmaceutical packaging. This module not only outperforms traditional 2D vision systems in defect capture, but also significantly improves data integrity and structural description accuracy.

[0035] In an embodiment of the present invention, the defect annotation module 200 aims to achieve high-robustness and high-precision primary annotation of areas with subtle depressions, indentations or structural anomalies through multi-scale spatial analysis and disturbance law extraction, thereby providing a reliable basis for subsequent defect identification and classification.

[0036] The method for identifying initial abnormal areas includes: extracting spatial micro-perturbation gradient values ​​between adjacent frames from a structured light depth map sequence and constructing a perturbation map; mapping the perturbation map into a directional perturbation intensity map using a directional projection operator; constructing a perturbation stability window of fixed size centered at each pixel point (x, y), calculating the rate of change of the directional perturbation within the window, and extracting an initial set of abnormal areas by setting a perturbation intensity threshold.

[0037] Specifically, the spatial perturbation gradient values ​​between adjacent frames are first extracted from the structured light depth map sequence. During structured light imaging, different frames correspond to different projection angles, thus generating subtle texture perturbations on the same surface under different illumination projections. These perturbations are visualized through pixel-level spatial gradient calculations. To avoid error accumulation due to projection center of gravity offset or optical distortion, the present invention uses the rate of change between consecutive frames as the basis for gradient differentiation.

[0038] After obtaining the perturbation map, a directional projection operator is introduced to map the perturbation map into a directional perturbation intensity map. This operation aims to eliminate the ambiguity of the perturbation direction caused by random illumination fluctuations or irregular projection structures.

[0039] Specifically, this operator decomposes the gradient vector of each pixel in the perturbation map by angle, counts the perturbation components in a certain direction, and constructs a directional perturbation intensity map. This can reveal the regularity of perturbation changes in a specific direction, effectively improving the stability and robustness of abnormal region extraction.

[0040] Specifically, when performing directional perturbation processing on the structural texture in the pharmaceutical packaging image, the perturbation does not rely on a fixed direction to determine defects. Instead, it performs edge perturbation response analysis based on four basic directions: 0°, 45°, 90°, and 135°, generating a structural response map for each direction. Subsequently, for the same position, the response intensity under each directional perturbation is calculated, and the direction with the maximum response value is selected as the optimal direction to construct a directional perturbation intensity map. This method has the ability to adapt to small defect structures at any angle. Even if the defects are not strictly arranged along the basic direction, the maximum response criterion can capture the perturbation information closest to the main direction, thereby realizing direction-adaptive texture anomaly recognition.

[0041] In addition, to further improve the robustness of direction selection, the system also introduces a redundant direction response judgment mechanism. If the response intensities in the two directions are similar (less than the set threshold difference), it is considered that there is an oblique or fuzzy defect structure. The two-way data is retained to participate in the subsequent defect credibility fusion, thereby enhancing detection accuracy.

[0042] Next, a perturbation stability window of fixed size (e.g., 5×5 or 7×7 pixels) is constructed with each pixel (x, y) as the center. The standard deviation of the directional perturbation intensity within the window is calculated as the perturbation change rate indicator. When the perturbation change rate of a window is significantly higher than the set threshold, the area is marked as an initial abnormal area.

[0043] This method enhances regional disturbance information through multi-directional projection, while simultaneously removing disturbances caused by background surface texture and reflections. This ensures that the extracted initial abnormal region has a clear spatial structure and stable disturbance characteristics. Unlike traditional static defect annotation methods based on grayscale or texture features, this method captures defect evolution from both temporal and directional dimensions, resulting in a higher level of anti-interference capability.

[0044] The generation of the preliminary defect marking map includes: performing contour tracking on the connected domain extracted from the initial abnormal area set to construct a multi-scale contour continuity factor; filtering and correcting the boundaries of the abnormal area based on the boundary closure and directional consistency at different scales, and outputting an optimized area set; using the corresponding positions of the optimized area set in the spatial coordinate depth map to construct a depression morphology feature map, extracting the average depression depth and disturbance sharpness in the area, combining with a preset depression judgment threshold, eliminating false alarm areas caused by packaging undulations or pattern interference, and outputting a defect confirmation candidate set; mapping all connected areas in the defect confirmation candidate set into binary mask images, and generating a preliminary defect marking map based on the area labels.

[0045] Specifically, first, contour tracking is performed on the connected domain extracted from the above-mentioned region set. The goal of contour tracking is to clarify the boundary shape of each abnormal region and provide basic data for subsequent boundary continuity analysis. The present invention introduces a multi-scale contour continuity factor to reconstruct the contour and calculate the closure of the region edge at different spatial resolutions (such as 1×, 2×, and 4× scaling). The contour continuity factor C is a weighted index obtained by combining the boundary closure rate and directional consistency. Through multi-scale contour analysis, the pseudo-defect areas with irregular boundaries and caused by background textures can be effectively filtered out, and the real concave areas with clear closed boundaries and consistent contour structures are retained.

[0046] After completing the boundary optimization, the process of constructing the concave morphology feature map begins. At the position of the structured light depth map corresponding to the optimized area set, the average concave depth and the disturbance sharpness coefficient of each area are extracted. The concave depth can be obtained by averaging the difference between the pixel depth value in the area and the average depth value of the surrounding area; and the disturbance sharpness coefficient is the ratio of the maximum disturbance gradient to the average disturbance gradient in the area, which is used to characterize the steepness of the area edge. It should be noted that the disturbance sharpness coefficient is particularly critical for distinguishing false alarm areas caused by normal packaging patterns or undulating textures, because real concave is usually accompanied by a sharp increase in edge disturbance, while the visual artifact area shows a soft transition or smooth undulation. By setting the depth threshold and the disturbance coefficient threshold, the abnormal areas are further screened and the areas that do not meet the concave judgment criteria are eliminated.

[0047] Finally, all retained connected domains in the defect confirmation candidate set are mapped into a binary mask image. This image marks defective areas at locations with a logical value of 1 and 0s elsewhere. Through a region label encoding mechanism, each defect is assigned a unique ID, and a preliminary defect labeling map is generated. This map not only contains information about the spatial distribution of defects but also possesses the structural standardization required for integration with subsequent modules, ensuring uniformity and traceability within the image processing pipeline.

[0048] In the preliminary defect marking module 200, the QR code structure region contains a highly dense information-encoding texture, making it highly susceptible to boundary artifacts due to missegmentation or reflection. Therefore, during the initial defect map generation phase, this region is temporarily shielded using a structural mask to prevent premature misjudgment. In the subsequent redundant block analysis module 300, to ensure structural integrity and logical verification capabilities of the QR code region, the system re-enables data access permissions for this region and performs operations including code block redundancy structure comparison, cross-frame scanning, and logical check matrix construction.

[0049] In an embodiment of the present invention, the redundant block parsing module 300 is used to perform structural analysis and redundant information reconstruction on the QR code region within the pharmaceutical packaging image, thereby achieving structural completion and reliable decoding of the QR code information even when initial defect detection fails. First, a structural block operation is performed on the target QR code region to extract a stable set of standard code blocks, which are then combined with the initial defect marker map to construct a spatial joint index. Based on this, if the initial recognition phase results indicate that the QR code region has defects such as occlusion, contamination, or reflections that cause initial decoding failure, the module will further perform cross-frame shifted sliding window scanning and perform frame-by-frame comparison of the redundant structures between code blocks to generate a logical check matrix.

[0050] Specifically, in the redundant structure parsing process of the two-dimensional code image, it is first necessary to perform structural block processing on the two-dimensional code area in the original image, aiming to divide the two-dimensional code image as a whole into multiple basic coding units with relatively stable geometric features and functional significance. The two-dimensional code structure is usually composed of multiple small modules (i.e., code blocks), each of which is a rectangular or square pixel matrix that carries specific coding information. Traditional image processing systems are mostly based on regional recognition, only performing bounding box recognition and ignoring the analysis of internal structures. The present invention, on the other hand, extracts the physical boundaries of each code block by jointly detecting edges and internal grid structures based on morphological gradient operators and directional mean filters, thereby realizing high-precision structural block segmentation of the two-dimensional code image. This structural block operation can not only eliminate false edge misidentification caused by background noise or light interference, but also further evaluate the clarity of each code block by introducing the pixel density mean variance index, thereby ensuring the reliability of the subsequent structure comparison process.

[0051] Subsequently, extracting a standard code block group is one of the fundamental operations for building a fault-tolerant QR code comparison model. This standard code block group is not directly extracted from a single frame image. Instead, it is formed by fusing code block units with high overlap, stable contrast, and clear pixel edges from multiple frames of images. This group is then used to determine a set of "standard structural models" through structural clustering and redundancy elimination strategies. This standard code block group can be considered an ideal structural reference in the QR code block space, encompassing key format identifiers, locators, version information areas, and data blocks within the QR code image. Establishing a standard code block group and encoding it as a unique index set helps improve matching accuracy during subsequent comparison and reconstruction processes.

[0052] To integrate the preliminary defect detection results with the code block structure, a three-dimensional hash mapping model was employed to establish a one-to-one mapping relationship between the pixel annotations in the preliminary defect marking image and the block units of the QR code structure. During this mapping process, a distance weighting function and a directional distribution correction term were introduced, ensuring accurate spatial region overlap even with a certain degree of perspective distortion or nonlinear image distortion. This joint indexing approach allows each structural code block to obtain its status in the defect marking image in real time, providing a positioning basis and a trusted framework for subsequent logical comparisons.

[0053] In the event of a preliminary decoding failure, the present invention does not directly enter the fault-tolerant compensation process, but instead performs a cross-frame sliding window redundant structure comparison: in a multi-frame image sequence, even if a frame fails to decode the QR code due to occlusion, blur, or reflection, the code block at the same position in other frames may still have a certain degree of readability. In particular, in scenarios where the camera has slight displacement or changes in viewing angle, the redundant structures in adjacent frames exhibit a high degree of spatial repetition. The present invention utilizes this phenomenon to perform structural matching of code blocks in the time dimension through a pixel-level sliding window mechanism, constructing a discriminant matrix for logical consistency.

[0054] The generation of the logic check matrix includes: performing pixel-level sliding window scanning with a certain code block as an initial window in a multi-frame image sequence, comparing the redundant structural differences between the sliding window area and the corresponding code blocks in adjacent frames, and selecting the frame with the strongest redundancy consistency to construct a code block check set; and establishing a two-dimensional logic check matrix based on the structural consistency comparison results of each code block in the code block check set with the standard code block group, wherein the elements represent the reliability level of each code block under the redundant structure verification.

[0055] Exemplarily, the sliding window mechanism uses each standard code block as a reference, and constructs a sliding window of a certain width (such as 8×8 pixels) with the position of the code block in the spatial coordinates in multiple frames as the center. Horizontal and vertical pixel shifts are performed on each frame, and the structural similarity between the window area and the reference code block is calculated in real time. The calculation method of structural similarity combines the histogram of oriented gradients (HOG) and two-dimensional Fourier texture features (2D-FFT). It not only focuses on the consistency of pixel grayscale values, but also evaluates whether the texture arrangement matches the boundary direction. By introducing a temporal continuity constraint function (i.e., the redundant structure of adjacent frames should not change suddenly), the sliding window mechanism can effectively skip the pseudo-matching area, retain the optimal redundant structure frame, and finally form a code block check set, that is, for a certain code block, it is a subset of pixels that show redundant structure consistency in multiple frames.

[0056] Based on this checksum, the system compares each redundant code block with the corresponding structure in the standard code block set. By calculating the local reconstruction rate and structural alignment confidence, it assigns a confidence score to each comparison result. Finally, the system integrates the comparison results of all code blocks into a two-dimensional array into a logical check matrix. Each element in this matrix represents the reliability level of a code block under the verification of the spatial redundancy structure. For example, a value of 1 indicates high consistency and recovery, a value of 0 indicates no reliable redundant structure, and a value of -1 indicates severe information loss and requires further intervention. This logical matrix provides a precise basis for subsequent error correction and regional reconstruction.

[0057] The generation of the logical check matrix not only summarizes the redundant comparison mechanism but also serves as the fundamental data for determining the successful reconstruction of each region of the QR code. This step begins with a sliding window scan centered on a specific code block. This operation, by sliding pixel by pixel, captures changes in the code block's perspective caused by subtle positional shifts between frames. Pixel-level scanning is more precise than traditional region-based sliding window operations and can detect even the slightest image changes. It is particularly well-suited for QR codes with regular structures and distinct blocky features.

[0058] Next, the process of selecting the frames with the strongest redundancy consistency is to screen and aggregate the above-mentioned sliding window scanning results. Only those frames that show structural stability with the reference code blocks in multiple feature dimensions (for example, symmetrical upper and lower edge strength, consistent distribution of corner points, and high regularity of module arrangement) will be included in the code block verification set. After the verification set is formed, the system will perform a multi-dimensional structural comparison with each code block in the standard code block group. The process includes structural contour fitting (using least squares fitting to determine the edge direction), information density analysis (determining whether the coding bits of the data module are complete) and logical offset correction (compensating for errors caused by image distortion during the shooting process). The comparison results are encoded as a set of structural confidence values ​​and input into the two-dimensional matrix unit to eventually form a complete logical check matrix.

[0059] The significance of this logical check matrix lies in providing a heat map judgment framework for the redundant structure of the QR code image. Each value in the matrix not only indicates whether the current code block is available, but also reveals the strategy priority of image fault tolerance processing through the redundant comparison strength level, laying a technical foundation for subsequent structural recovery and decoding optimization.

[0060] In the embodiment of the present invention, the defect credibility fusion module 400 specifically includes:

[0061] The fusion process is to perform pixel-by-pixel logical weighted fusion on the preliminary defect marking image and the logic check matrix to form a joint response image.

[0062] Specifically, the pixel-by-pixel logical weighted fusion here refers to the weighted superposition processing of the binary defect value in the preliminary defect marking image and the code block reliability level value in the logical check matrix at each corresponding pixel point. Conventional image fusion methods mostly use addition or Boolean logic (such as AND / OR) for simple superposition, which is difficult to reflect the relative weight and credibility of multi-source information. The fusion method in the present invention introduces a logical weighting factor, and sets the corresponding weight coefficient according to the redundant consistency level in the logical check matrix, weakening the mislabeled areas that may exist in the preliminary marking, while strengthening the defect response in the area with high redundant consistency. For example, if a pixel is a suspected defect in the preliminary marking image (value is 1), and its corresponding code block reliability level in the logical matrix is ​​high (value close to 1), then the response value of the point after fusion is close to 1, indicating that the point is more likely to be a credible defect; otherwise, the response of the point is weakened. This operation significantly improves the credibility expression ability of the final response map.

[0063] Based on the joint response map, the average response intensity of each connected area is calculated as a credibility indicator, and the values ​​of all pixel positions are expanded into a credibility map for the entire area. This operation aims to achieve a unified credibility mapping from a spatial level, so that subsequent threshold elimination operations can more accurately identify and classify regions. Generally speaking, if threshold processing is performed directly at the pixel level, edge pixels may be misjudged or missed. The present invention effectively reduces edge interference by assigning consistent credibility to the entire area, making the elimination strategy more stable and holistic; eliminate all areas below the credibility threshold to obtain a final defect confirmation map; if the final defect confirmation map covers an area containing a QR code structure, and the original QR code decoding fails, the area in the final defect confirmation map that overlaps with the QR code area is marked as a structural damage defect; prompting that structural repair is required; otherwise, the final defect confirmation map is output as the final defect detection result.

[0064] It should be noted that the connected area refers to the continuous defect response area formed by connecting adjacent pixels (usually 8 adjacent) in the image. These areas usually represent potential defects such as cracks, stains or missing QR code information in the actual image. The statistical average response intensity can effectively avoid excessive defect responses caused by local strong noise or mislabeling, and thus evaluate whether the area is a real defect based on the overall intensity of the area. This process reflects that the present invention takes dual considerations of defects at the "point" and "surface" levels, overcomes the problem that traditional point recognition methods are susceptible to local disturbances, and improves the overall robustness of defect credibility assessment.

[0065] Furthermore, if the defective area in the final defect confirmation diagram covers the QR code structure area, and the QR code fails to be decoded in the initial decoding stage (i.e., in the operation of module 200), it is inferred that the QR code suffers structural damage, and the overlapping area is further marked as a structural damage defect, and a structural repair prompt is triggered. In the technical logic of the present invention, the QR code is not only a key element of information recognition, but also serves as the basis for judging the integrity of the image structure. Therefore, its decoding result has a special status in credibility fusion. If the decoding failure only occurs in the suspected defective area, it is highly indicated that the area interferes with the QR code encoding information and is highly destructive. Therefore, the present invention uses this situation to determine structural defects and makes repair suggestions, such as reshooting, image enhancement, or manual verification. If the above conditions are not met, that is, the QR code area is not covered in the final confirmation diagram, or the QR code is successfully decoded, the system directly outputs the final defect confirmation diagram as the final defect detection result for subsequent packaging screening, early warning, or traceability.

[0066] The final result here is highly accurate, has eliminated redundant interference and combined with multi-source structural consistency analysis, and can be widely used in industrial visual inspection systems to achieve real-time and reliable defect recognition and information closed-loop processing.

[0067] In summary, a machine vision-based pharmaceutical packaging defect detection system based on an embodiment of the present invention is illustrated, which achieves accurate restoration of the surface morphology of pharmaceutical packaging through high-precision three-dimensional imaging technology based on structured light coding, combined with multi-angle parallax reconstruction and multi-channel perturbation analysis, and constructs a depth gradient difference map and a perturbation map on this basis, effectively capturing tiny abnormal features such as depression-type defects and structural perturbations, thereby improving the system's perception of fine-grained defects. By constructing a redundant block parsing mechanism, when the QR code area is occluded or damaged, the cross-frame redundant structure can still be used for content restoration and logical verification, effectively reducing the risk of misjudgment caused by failure of QR code decoding, and improving the robustness of the system in multiple types of interference scenarios. In addition, the present invention also has a credibility fusion mechanism, which dynamically generates a credibility response map by combining defect labeling results and redundant verification information, and eliminates low-credibility areas, thereby significantly improving the accuracy of defect confirmation.

Claims

1. A drug packaging defect detection system based on machine vision, characterized in that: include: The structured light coding acquisition module uses Gray code structured light stripes to project multi-angle codes onto the surface of the drug packaging. It uses a high-resolution camera to capture a sequence of stripe deformation images and reconstructs a spatial coordinate depth map based on the differences in multi-frame projections. The defect annotation module calculates the depth gradient difference of the spatial coordinate depth map and constructs a disturbance map to identify the initial abnormal area; Suspicious concave areas are extracted based on the preset threshold in the disturbance map, and primary defects are marked based on the continuity of the contour boundary to generate a preliminary defect marking map; The redundant block parsing module performs structural block segmentation on the QR code area in the pharmaceutical packaging image, extracts a standard code block group, and constructs a spatial index that is jointly matched with the preliminary defect mark map. When the preliminary defect mark map covers the QR code area and the QR code fails to be decoded initially, a cross-frame shifted sliding window scan and code block redundant structure comparison are performed to generate a logical check matrix. The defect credibility fusion module fuses the preliminary defect marking map with the logic check matrix, generates the final defect confirmation map based on the credibility parameters, and performs drug packaging defect detection.

2. The machine vision-based drug packaging defect detection system according to claim 1, characterized in that: The multi-angle coding projection includes: using a multi-frequency Gray code structured light coding method to construct a coding sequence set composed of a high-frequency fine coding subsequence and a low-frequency coarse coding subsequence; projecting the coding sequence onto the surface of the drug packaging at multiple fixed angles, and synchronously recording the corresponding projection image sequence at each angle, while establishing an angle association index matrix to mark the spatial correspondence.

3. The machine vision-based drug packaging defect detection system according to claim 2, characterized in that: The method for reconstructing a spatial coordinate depth map based on multi-frame projection differences includes: performing phase difference processing between adjacent frames of a projection image sequence to construct a multi-channel fringe disturbance map to describe the distribution of fringe offsets caused by surface topography changes; performing pixel-level joint fitting using multi-angle channels corresponding to the multi-channel fringe disturbance map to reconstruct a joint disparity map, and extracting continuous topography gradients through sub-pixel interpolation technology; and converting the joint disparity map into a spatial coordinate depth map, which is output as a sequence of structured light depth maps.

4. The machine vision-based drug packaging defect detection system according to claim 1, characterized in that: The method for identifying initial abnormal areas includes: extracting spatial micro-perturbation gradient values ​​between adjacent frames from a structured light depth map sequence and constructing a perturbation map; mapping the perturbation map into a directional perturbation intensity map using a directional projection operator; constructing a perturbation stability window of fixed size centered at each pixel point (x, y), calculating the rate of change of the directional perturbation within the window, and extracting an initial set of abnormal areas by setting a perturbation intensity threshold.

5. The machine vision-based drug packaging defect detection system according to claim 4, characterized in that: The generation of the preliminary defect marking map includes: performing contour tracking on the connected domain extracted from the initial abnormal area set to construct a multi-scale contour continuity factor; filtering and correcting the boundaries of the abnormal area based on the boundary closure and directional consistency at different scales, and outputting an optimized area set; using the corresponding positions of the optimized area set in the spatial coordinate depth map to construct a depression morphology feature map, extracting the average depression depth and disturbance sharpness in the area, combining with a preset depression judgment threshold, eliminating false alarm areas caused by packaging undulations or pattern interference, and outputting a defect confirmation candidate set; mapping all connected areas in the defect confirmation candidate set into binary mask images, and generating a preliminary defect marking map based on the area labels.

6. The machine vision-based drug packaging defect detection system according to claim 1, characterized in that: The generation of the logic check matrix includes: performing pixel-level sliding window scanning with a certain code block as an initial window in a multi-frame image sequence, comparing the redundant structural differences between the sliding window area and the corresponding code blocks in adjacent frames, and selecting the frame with the strongest redundancy consistency to construct a code block check set; and establishing a two-dimensional logic check matrix based on the structural consistency comparison results of each code block in the code block check set with the standard code block group, wherein the elements represent the reliability level of each code block under the redundant structure verification.

7. The machine vision-based drug packaging defect detection system according to claim 1, characterized in that: The fusion process is to perform pixel-by-pixel logical weighted fusion on the preliminary defect marking image and the logic check matrix to form a joint response image.

8. The machine vision-based drug packaging defect detection system according to claim 7, characterized in that: Based on the joint response map, the average response intensity of each connected area is calculated as the credibility indicator. The values ​​of all pixel positions are expanded to form the credibility map of the entire area. All areas below the credibility threshold are eliminated to obtain the final defect confirmation map. If the coverage area of ​​the final defect confirmation image includes the QR code structure area and the original QR code decoding fails, the area overlapping with the QR code area in the final defect confirmation image will be marked as a structural damage defect; It prompts that structural repair is needed; otherwise, the final defect confirmation diagram is output as the final defect detection result.

9. A computer-readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the machine vision-based pharmaceutical packaging defect detection system according to any one of claims 1 to 8 is implemented.

10. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the machine vision-based pharmaceutical packaging defect detection system as described in any one of claims 1 to 8.

Images