Method for on-line detection of defects in plastic articles based on machine vision

Abstract

The application discloses a plastic product flaw online detection method based on machine vision and belongs to the technical field of machine vision detection. The method comprises the following steps: acquiring continuous multiple surface images of a plastic product, calculating local binary pattern texture features and histogram of oriented gradients edge features of each image respectively, weighting and fusing the two features according to pixel positions to generate a fused feature response map, performing adaptive threshold segmentation on the fused feature response map to obtain initial flaw seed points, taking the initial flaw seed points as growth starting points, performing region growing by using four-neighbor connectivity and feature similarity constraint to obtain a connected flaw region, and performing boundary smoothing processing and hole filling on the connected flaw region to output a final flaw detection result. The method realizes high-precision online identification of surface flaws of the plastic product by means of multi-feature fusion and an adaptive region growing mechanism.

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection technology, specifically to an online defect detection method for plastic products based on machine vision. Background Technology

[0002] In the production of plastic products, automated online detection of surface defects is a crucial step in ensuring product quality. Existing machine vision inspection methods typically rely on single-type image features for defect identification, such as using texture analysis or edge detection alone to identify abnormal areas like scratches, stains, and bubbles. This single-feature-based approach is prone to insufficient feature representation capabilities when dealing with the complex surface conditions of plastic products. When the product surface simultaneously contains areas of high gloss reflection, transparent structures, or frosted textures, single texture features are sensitive to subtle color differences but weakly respond to defects with blurred shapes. Single edge features can capture abrupt contour changes but easily miss texture abnormalities caused by gradual changes in surface roughness. These two types of features form isolated information silos, limiting the comprehensiveness of the detection.

[0003] The shortcomings of existing technical solutions also lie in the extraction and growth strategies for suspected defective regions. Conventional detection processes typically use a fixed empirical threshold to hard segment the feature map to identify defective pixels. This approach ignores the fluctuations in background features across different regions of the image. A fixed threshold may miss minute defects in areas with weak background texture, while generating a large amount of false defect noise in areas with strong background texture. In the subsequent pixel region aggregation stage, conventional methods either directly connect all out-of-limit pixels or use unconstrained region growth, lacking a growth rule dynamically bound to local feature similarity. This results in inaccurate boundaries of the grown defective regions, easily incorporating adjacent normal textures, or causing the same defect to be fragmented into multiple pieces due to discontinuous local responses. Summary of the Invention

[0004] The purpose of this invention is to provide an online detection scheme that can adapt to the surface characteristics of various plastic materials and possesses high detection accuracy and strong robustness for various defect morphologies. This scheme addresses the problem of incomplete representation of complex surface defects by single image features. It improves the unified response capability to multiple types of defects by constructing complementary feature representations that integrate texture and edge information. Simultaneously, it addresses the issues of low defect extraction accuracy and poor region integrity caused by fixed threshold segmentation and unconstrained region growth. It establishes an adaptive seed point selection mechanism and growth constraint rules that are linked to the statistical characteristics of local image features, ensuring that the final output defect region boundaries are smooth and complete and highly consistent with real defects.

[0005] To achieve the above objectives, the present invention provides the following technical solution: The present invention provides an online defect detection method for plastic products based on machine vision, comprising the following steps: This method acquires multiple consecutive frames of surface images of a plastic product. For each frame, local binary pattern texture features and histogram-of-direction (HCD) edge features are calculated. These features are then weighted and fused according to pixel position to generate a fused feature response map. Adaptive threshold segmentation is applied to the fused feature response map to obtain initial defect seed points. Using these initial seed points as the starting point, region growth is performed using four-neighbor connectivity and feature similarity constraints to obtain connected defect regions. Boundary smoothing and hole filling are then applied to these connected defect regions, and the final defect detection result is output. This method, by fusing texture and edge features and employing adaptive segmentation and growth, can capture subtle surface defects of plastic products online and eliminate boundary interference and internal holes while obtaining a complete defect outline, significantly improving detection accuracy.

[0006] As a preferred technical solution of the present invention, when acquiring continuous multi-frame surface images, multiple industrial cameras simultaneously acquire images of the plastic product surface from different angles, with each industrial camera outputting an image sequence at a fixed frame rate. The images acquired by all cameras at the same time are subjected to a viewpoint normalization transformation based on calibration parameters, projecting the images from different viewpoints onto a unified spatial reference plane. On this unified spatial reference plane, phase correlation matching is performed on the normalized images at adjacent times to compensate for the positional offset of the plastic product on the conveyor belt, obtaining aligned continuous multi-frame surface images. Specifically, the phase correlation matching uses Fourier transform to calculate the translation amount between adjacent frames, and performs reverse translation compensation on the subsequent frame image based on this translation amount, thereby eliminating image misalignment caused by conveyor motion and ensuring the accuracy of spatial registration of the multi-frame images.

[0007] Furthermore, each industrial camera's lens is equipped with a ring polarizer and a bandpass filter. The ring polarizer eliminates specular reflections from the surface of plastic products, and the center wavelength of the bandpass filter matches the dominant wavelength of the illumination source to suppress ambient light interference. Before image acquisition, camera response consistency calibration is performed, recording the pixel response curve of each camera under a uniform white reference plate. These response curves are then used to perform pixel-by-pixel gain and bias correction on each subsequently acquired frame, ensuring high consistency in brightness and response between the multiple cameras, thus providing stable and reliable images for subsequent feature extraction.

[0008] As another preferred embodiment of the present invention, the specific implementation of extracting texture and edge features and performing weighted fusion is as follows: For each pixel in each frame of the image, the gray-level difference of its sampling points on the neighborhood ring is calculated to generate a local binary pattern code, and the histogram distribution of the code is statistically analyzed as the local binary pattern texture feature value of the pixel; the gradient direction within the local window where the pixel is located is quantized and voted to construct an directional gradient histogram, and the magnitude of the main gradient direction is taken as the directional gradient histogram edge feature value of the pixel; the local binary pattern texture feature value is multiplied by a first weight coefficient, and the directional gradient histogram edge feature value is multiplied by a second weight coefficient, and the two are added together to obtain the fused feature value of the pixel; the fused feature response map is generated by traversing all pixels in the image. Weighted fusion of texture features and edge features can enhance the comprehensive response to various types of defects such as scratches, shrinkage marks, and stains on the surface of plastic products, avoiding missed detections caused by single features.

[0009] Preferably, the first weighting coefficient and the second weighting coefficient are dynamically adjusted according to the surface material type of the plastic product: when the surface of the plastic product is a glossy material, the first weighting coefficient is set to a first value and the second weighting coefficient is set to a second value, wherein the first value is greater than the second value to highlight texture abnormalities; when the surface of the plastic product is a frosted material, the first weighting coefficient is set to a third value and the second weighting coefficient is set to a fourth value, wherein the third value is less than the fourth value to enhance edge contrast; when the surface of the plastic product is a transparent material, the first weighting coefficient is set to a fifth value and the second weighting coefficient is set to a sixth value, and the ratio of the fifth value to the sixth value is taken as the reciprocal of the haze value of the transparent material, thereby adapting to the optical characteristics of the transparent material, so that the fused features can maintain the best distinction between defects and background on different materials.

[0010] As a preferred embodiment of the present invention, the step of adaptive threshold segmentation of the fused feature response map to obtain initial defect seed points specifically includes: statistically analyzing the distribution of all pixel values ​​in the fused feature response map, calculating the mean and standard deviation of the entire response map; adding the mean and standard deviation of the entire map by a preset multiple to obtain a first dynamic threshold, and subtracting the standard deviation by a preset multiple from the mean to obtain a second dynamic threshold; marking pixels with pixel values ​​greater than the first dynamic threshold as high anomalies, and marking pixels with pixel values ​​less than the second dynamic threshold as low anomalies; and merging the high anomalies and low anomalies as initial defect seed points. Based on a global statistical dual-threshold mechanism, this approach can simultaneously capture abnormal regions above and below the background, effectively addressing defects of different grayscale polarities such as white scratches and black impurities, while avoiding the influence of lighting fluctuations on fixed thresholds.

[0011] More preferably, the preset multiple of the standard deviation is adaptively adjusted based on the frequency of defects occurring in plastic products across multiple consecutive frames of images: The previous detection results of the current batch of plastic products are statistically analyzed, and the historical average of the deviation multiple of the defect area relative to the mean in the fused feature response map is calculated. Using this historical average as a benchmark multiple, if the number of initial defect seed points detected in the current frame is greater than a preset proportion of the historical average seed point number, the benchmark multiple is increased by a step value to obtain an increased preset multiple; if the number of initial defect seed points detected in the current frame is less than a preset proportion of the historical average seed point number, the benchmark multiple is decreased by a step value to obtain a decreased preset multiple. This automatically tightens the threshold when too many suspected seed points are generated, suppressing false detections due to noise, and automatically widens the threshold when too few suspected seed points are generated, avoiding missed detections of minor defects, thus achieving online self-adjustment of detection sensitivity.

[0012] As a preferred technical solution of the present invention, the process of obtaining a connected defect region by using the initial defect seed point as the growth starting point and utilizing four-neighbor connectivity and feature similarity constraints for region growth is as follows: An empty set of growth regions is established, and all initial defect seed points are added to the growth queue; a seed point is taken from the growth queue, and its four adjacent pixels (up, down, left, and right) are searched; for each adjacent pixel, the absolute difference between the fused feature value of the adjacent pixel and the fused feature value of the current seed point is calculated. If the absolute difference is less than a preset similarity threshold and the adjacent pixel is not marked as grown, then the adjacent pixel is added to the growth region and simultaneously added to the growth queue; the above extraction and search steps are repeated until the growth queue is empty, and the current growth region is stored in the connected defect region set. This growth strategy uses feature similarity as a consistency constraint, which can expand the initial discrete seed points into a connected domain that matches the actual defect boundary, overcoming the defect region breakage caused by threshold segmentation alone.

[0013] Furthermore, the preset similarity threshold is dynamically determined based on the texture fluctuation range of the background area on the surface of the plastic product: the variance of all non-seed point pixel values ​​in the fused feature response map is statistically analyzed, and the square root of the variance is used as the standard deviation of the background fluctuation; the standard deviation of the background fluctuation is multiplied by a preset scaling factor to obtain the dynamic similarity threshold. By adaptively adjusting the growth tolerance through background fluctuation, the similarity condition can be appropriately relaxed on matte surfaces with strong texture to prevent undergrowth, while strict similarity constraints are maintained on uniform high-gloss surfaces to avoid growing normal textures into defects and improve the accuracy of region growth.

[0014] As another preferred embodiment of the present invention, the steps of performing boundary smoothing and hole filling on the connected defect regions and outputting the final defect detection result are as follows: For each connected defect region, extract its contour pixel sequence, and perform iterative dilation and erosion operations on the contour pixel sequence using morphological closing operations to eliminate burrs and depressions on the contour; convert the connected defect regions after boundary smoothing into a binary mask image, and detect internal closed holes not covered by defect regions in the binary mask image; for each detected closed hole, calculate the Euclidean distance between the hole boundary and the boundary of the adjacent defect region; if the Euclidean distance is less than a preset distance threshold, fill the hole pixel as part of the defect region; merge and output all connected defect regions after hole filling as the final defect detection result. Closing operations can make the defect boundary continuous and smooth, eliminating fine jagged edges, while hole filling based on distance conditions can repair internal breaks in the defect, restore the true defect shape, and provide complete morphological information for subsequent defect level determination.

[0015] The technical effects and advantages provided by the present invention in the above technical solution are as follows: Local binary pattern texture features and histogram-oriented gradient (HARG) edge features are weighted and fused at pixel positions to generate a fused feature response map. Local binary pattern texture features, by statistically analyzing the gray difference distribution in the pixel neighborhood, can reliably capture subtle changes in surface roughness and periodic texture breaks. HARG edge features, through the quantized accumulation of gradient directions within a local window, enhance the response to defect contours and shape boundaries. After these two types of features are weighted and superimposed at the pixel level, the resulting fused feature response map carries both texture anomaly information and edge abrupt change information. Low-contrast scratches, which originally had a weak response in a single feature domain, have their edge characteristics highlighted in the fused response; discoloration spots, which originally lacked clear shape boundaries, have their anomalous texture response preserved in the fused map. This fusion method allows a single detection process to handle various types of defects such as scratches, discoloration spots, shrinkage marks, and bubbles in parallel, without needing to maintain multiple detection channels for different defect types, thus avoiding missed detections or feature blind spots caused by single features on complex surface materials.

[0016] Starting with the initial defect seed point, region growth is performed using four-neighbor connectivity and feature similarity constraints to obtain connected defect regions. The growth process uses the defect seed point as an anchor point, searching only among the four adjacent pixels (top, bottom, left, and right) to avoid disordered spread along the diagonal. The growth acceptance condition is strictly bound to the absolute difference between the fused feature value of the adjacent pixel and the fused feature value of the current seed point. Neighboring pixels are only included in the growth region when this difference is less than a dynamically determined similarity threshold. This similarity threshold is dynamically calculated based on the variance of all non-seed point pixel values ​​in the fused feature response map, ensuring that the growth rules are consistent with the texture fluctuation range of the current image background. When the background texture consistency is high, the similarity threshold is automatically tightened, resulting in sharp boundaries for the grown defect region that do not erode adjacent normal regions. When the background texture is highly cluttered, the similarity threshold is automatically widened to prevent premature termination of growth due to small fluctuations in local response values, thus reconnecting the parts of a complete defect that were previously separated by weak response bands into a unified region. This growth constraint, which is linked to the level of feature fluctuations, improves both the spatial coherence and boundary fit of the output connected defect regions. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0018] Figure 1 This is a flowchart of an online defect detection method for plastic products based on machine vision; Figure 2 This is a flowchart of a method for multi-camera image acquisition and phase-correlation motion compensation on the surface of plastic products; Figure 3 This is a schematic diagram of the process for generating the fused feature response map; Figure 4 It is a flowchart of the process for smoothing the boundaries of connected defective areas and filling internal holes. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] See Figure 1This invention provides an online defect detection method for plastic products based on machine vision, comprising the following steps: acquiring multiple consecutive frames of surface images of the plastic product; calculating local binary pattern texture features and histogram of orientation gradients (HARGRs) edge features for each frame; weighting and fusing the local binary pattern texture features and HARGR edge features according to pixel position to generate a fused feature response map; performing adaptive threshold segmentation on the fused feature response map to obtain initial defect seed points; using the initial defect seed points as the growth starting point, performing region growth using four-neighbor connectivity and feature similarity constraints to obtain connected defect regions; performing boundary smoothing and hole filling on the connected defect regions, and outputting the final defect detection result.

[0021] In specific implementation, please refer to Figure 2 Multiple industrial cameras are mounted at different positions on the plastic product conveyor belt, each synchronously outputting image sequences at a fixed frame rate. Each camera lens is equipped with a circular polarizer and a bandpass filter. The circular polarizer weakens the highlights caused by specular reflection on the plastic product surface, while the center wavelength of the bandpass filter matches the dominant wavelength of the illumination source, which is a narrow-band wavelength selected based on the plastic product material. By adjusting the rotation angle of the circular polarizer, components of the light reflected from the plastic product surface with polarization directions orthogonal to the polarizer's transmission direction are suppressed, thus eliminating specular reflection. The bandpass filter only allows light near the center wavelength to pass through, suppressing stray ambient light.

[0022] Before image acquisition, camera response consistency calibration is performed. A uniform white reference plate is placed on the conveyor belt, covering the entire field of view of the industrial cameras. Illumination is applied, allowing all industrial cameras to synchronously acquire multiple frames of images of the white reference plate under different exposure conditions. For each pixel location of each industrial camera, the grayscale value output by the pixel under different exposure levels is recorded, and a linear response model is obtained by fitting using the least squares method. In the linear response model, the pixel response curve expresses the linear relationship between the pixel grayscale value and the input radiant energy. For the pixel coordinates of the m-th industrial camera... Calculate the gain coefficient corresponding to the pixel position. and bias coefficient This normalizes the response of the pixel to a uniform reference response line. During the actual acquisition of each frame of image, gain and bias correction are performed pixel-by-pixel on the raw image output from each industrial camera. The corrected image pixel values ​​are: in: These are the original pixel values. After this correction, all industrial cameras will output consistent grayscale values ​​for the same radiance.

[0023] Camera calibration parameters are obtained using a stereo calibration plate. All industrial cameras simultaneously capture images of the calibration plate in different poses, and the intrinsic parameter matrix and extrinsic parameter rotation matrix and translation vector parallel to the conveyor belt plane are calculated for each camera. For multi-view images acquired at the same time, distortion correction is performed on each image using the intrinsic parameters. Then, based on the extrinsic parameters, the distortion-corrected images are projected onto a unified spatial reference plane with the conveyor belt plane as the reference, generating normalized images for each viewpoint on this unified spatial reference plane. The unified spatial reference plane is defined as the spatial plane containing the conveyor belt surface; all projected normalized images have the same spatial resolution and coordinate system origin.

[0024] On a unified spatial reference plane, phase correlation matching is performed on the normalized images of two adjacent time points to compensate for the positional shift of the plastic product during the conveyor belt movement. Let the normalized image of the previous time point be... The normalized image at the next time step is , where x and y are the abscissa and ordinate on the unified spatial reference plane, respectively, in pixels. Calculate respectively. and The two-dimensional discrete Fourier transform yields the spectrum. and Let u and v be the spatial frequency variables in the x and y directions, respectively. The normalized cross-power spectrum is calculated using the following formula. : in, express The complex conjugate, This represents taking the modulus of a complex number, where the numerator is the pointwise product of the two spectra, and the denominator is the modulus of the product. Calculate... The two-dimensional discrete Fourier inverse transform is used to obtain the two-dimensional impulse function in the spatial domain. .exist The coordinates of the maximum translation value are searched, and the horizontal coordinate value of this coordinate is recorded as the horizontal component of the translation. The vertical component of the translation is denoted by the ordinate value. , and This represents the number of pixels the subsequent frame is offset from the previous frame in the x and y directions. Based on the translation amount... For images Perform reverse translation compensation to generate an aligned image. Reverse translation compensation is achieved by... The pixel values ​​shift as a whole To achieve this, missing boundary regions are filled with black pixels or the original edge values ​​before translation are preserved. The phase correlation matching and reverse translation compensation operations are repeated for all adjacent frames in a series of normalized images to obtain spatially aligned surface images of consecutive frames.

[0025] In specific implementation, please refer to Figure 3 For each frame of a series of aligned surface images, local binary pattern texture features and histogram edge features are extracted pixel by pixel. The image is converted from color space to grayscale space. If the input image is a multi-channel image, the weighted average of the red, green, and blue channels is taken as the grayscale value. The weighting coefficients are usually determined based on the brightness perception standard.

[0026] For each pixel in the image, a ring with a radius of R pixels is defined centered on that pixel. P sampling points are sampled at equal intervals on the ring, where R takes the value of 1, 2, or 3, and P takes the value of 8, 16, or 24. When the coordinates of a sampling point do not fall on an integer pixel position, bilinear interpolation is used to calculate the gray value of that sampling point. The gray values ​​of the P sampling points on the ring are compared one by one with the gray value of the center pixel. If the gray value of a sampling point is greater than or equal to the gray value of the center pixel, the corresponding encoding position of that sampling point is recorded as 1; otherwise, it is recorded as 0, generating a P-bit binary code. This P-bit binary code is the local binary pattern code for the center pixel. The P-bit binary number is converted into a decimal number in a fixed clockwise or counterclockwise direction. This decimal number is used as the local binary pattern code value for that pixel. The local binary pattern code value is then subjected to rotation-invariant mapping, and the smallest decimal value after cyclic shifting the P-bit binary code is taken as the rotation-invariant local binary pattern code value. Analyze the histogram distribution of rotation-invariant local binary pattern (LoBMM) encoded values ​​for all pixels within an M×M pixel square neighborhood window centered on the given pixel. M can be 32, 48, or 64, and the number of bins in the histogram corresponds to the number of possible values ​​for the LIBMM encoded value. Use the frequency of the bin with the highest count in the histogram as the LoBMM texture feature value for that central pixel. Perform this operation on every pixel in the image to generate a LoBMM texture feature map.

[0027] For the same grayscale image, for each pixel, an N×N local window is taken around it, where N can be 32, 48, or 64. Within this local window, the horizontal gradient component of each pixel is calculated using the central difference operator. and vertical gradient components , where p and q are the row and column indices of the pixels within the local window, respectively, calculated with the top-left corner of the local window as the origin. According to and Calculate the gradient magnitude of the pixel using the following formula. and gradient direction angle : in, The horizontal gradient component represents the rate of change of gray level of a pixel in the horizontal direction. The vertical gradient component represents the rate of change of gray level of a pixel in the vertical direction. The gradient magnitude represents the degree of drastic change in the grayscale of that pixel. The gradient direction angle represents the angle between the direction of the most drastic grayscale change at a pixel and the horizontal direction, expressed in radians. arrive The gradient direction angle interval is uniformly divided into K direction intervals, where K takes the value of 8, 9, or 12, and each direction interval covers Radius. For each pixel within the local window, its gradient direction angle is calculated. Project the gradient value of the pixel into the corresponding direction range and then project it into the corresponding direction range. The statistical values ​​of this directional interval are accumulated as voting weights to construct the histogram of directional gradients for the local window. The directional interval with the largest statistical value is then identified within the histogram, and this largest statistical value is used as the edge feature value of the histogram of directional gradients for the center pixel of that local window. This process is repeated for each pixel in the image to generate the edge feature map of the histogram of directional gradients.

[0028] For local binary pattern texture features and histogram edge features at the same pixel location in the same frame, a fused feature value is calculated using a weighted fusion method. The local binary pattern texture feature value is then multiplied by a first weighting coefficient. The edge feature values ​​of the directional gradient histogram are multiplied by the second weighting coefficient. The two values ​​are added together to obtain the fused feature value of the pixel. The entire image is then traversed, and the fused feature values ​​of each pixel are arranged according to their original spatial positions to generate a fused feature response map.

[0029] In the weighted fusion process, the first weight coefficient Second weighting coefficient The adjustment is dynamically based on the surface material type of the plastic product. The material type of the plastic product is obtained through configuration parameters on the host computer or automatic determination by the material recognition sensor. The material types are divided into three categories: high-gloss material, frosted material, and transparent material.

[0030] When the surface of a plastic product is a high-gloss material, the surface reflection is strong, and the specular reflection component obscures some texture details. In this case, the discrimination weight of texture features should be increased. The first weight coefficient should be... Set the first value to 0.8, and the second weighting coefficient. The second value is set to 0.2, and the first value of 0.8 is greater than the second value of 0.2. The reason for setting the first value of 0.8 and the second value of 0.2 is that, under the glossy surface material, the texture features have a higher degree of distinguishing defects than the edge features. By increasing the weight of the texture features, the false detection of edges caused by specular reflection can be suppressed.

[0031] When the surface of a plastic product is made of a frosted material, the surface is rough, diffuse reflection dominates surface imaging, and texture information is highly random, while edge features respond more stably to defects such as scratches and dents. The first weighting coefficient... Set the third value to 0.3, the second weighting coefficient. The fourth value is set to 0.7, and the third value of 0.3 is less than the fourth value of 0.7. The reason for setting the third value of 0.3 and the fourth value of 0.7 is that, under the frosted material, the edge features are better at capturing small surface deformations than the texture features, and increasing the weight of the edge features is beneficial to improving the contrast between imperfections and the background.

[0032] When the surface of a plastic product is transparent, the haze value of the transparent material is defined as the percentage ratio of the scattered light flux deviating from the incident direction by more than 2.5 degrees to the total transmitted light flux, denoted by the symbol [symbol missing]. This indicates the haze value. The value range is greater than 0 and less than or equal to 100. The first weighting coefficient... Set to the fifth value Second weighting coefficient Set to the sixth value The fifth value With the sixth value The fifth value satisfies the following relationship. With the sixth value The ratio is equal to the haze value of the transparent material. The reciprocal of, is expressed as At the same time, constraints In transparent materials, higher haze values ​​result in stronger surface scattering and reduced reliability of texture features. By dynamically adjusting the ratio of texture features to edge features using the reciprocal of the haze value, the weight of texture features decreases as the haze value increases, while the weight of edge features increases. For example, when the haze value of a transparent material is 5, the fifth value... With the sixth value The ratio is 0.2, and the solution obtained by combining the normalization constraints is... , .

[0033] In practice, after obtaining the fused feature response map, the distribution of all pixel values ​​in the map is statistically analyzed. Let the size of the fused feature response map be H rows and W columns, and the fused feature values ​​of all pixels constitute a dataset containing H×W values. The mean of this dataset is calculated, denoted by the sign... This indicates the mean of the entire graph. This is the arithmetic mean of the fused feature values ​​for all pixels. The standard deviation of this dataset is also calculated, denoted by the sign... Indicates standard deviation The feature values ​​of each pixel are fused with the mean of the entire image. The square root of the arithmetic mean of the squares of the differences.

[0034] Let the pre-defined multiple of the standard deviation be denoted by the sign. Indicates the preset multiple. For each real number greater than zero, the mean of the entire graph is... Preset multiple of standard deviation The products are added together to obtain the first dynamic threshold, which is represented by the symbol... It means, that is Measure the average of the entire graph. Subtract the preset multiple of the standard deviation The product of these values ​​yields the second dynamic threshold, which is represented by the symbol... It means, that is .

[0035] Iterate through each pixel in the fused feature response map. If the fused feature value of a pixel is greater than the first dynamic threshold... If the fused feature value of a pixel is less than the second dynamic threshold, then that pixel is marked as a high anomaly. If a pixel is found to be abnormally bright in the fused feature response map, it is marked as a low anomaly. High anomalies correspond to abnormally bright areas in the fused feature response map, while low anomalies correspond to abnormally dark areas. Both types of anomalies represent the deviation of the plastic product surface from the normal background area. All high anomalies and all low anomalies are merged into a single set, and all pixels in this set are used as initial defect seed points.

[0036] Preset multiple of standard deviation The system adaptively adjusts based on the frequency of defects appearing in multiple consecutive frames of images of plastic products. During system operation, it continuously records the detection results of each frame for the current batch of plastic products and calculates the pixel value of the defective region in each frame relative to the overall image mean in the fused feature response map. The deviation factor is defined as the ratio of the fused feature value to the mean value of the entire image for each pixel within the defective region. The absolute value of the difference divided by the standard deviation The quotient value is used to average the quotient values ​​of all pixels in the defective areas, and this average is taken as the actual deviation factor of the frame image. The actual deviation factors of all frames processed in the current batch are averaged again to obtain the historical average, denoted by [symbol missing]. This indicates the historical average. As a benchmark multiple.

[0037] Simultaneously, the number of initial defect seed points detected in each frame of all frames processed in the current batch is recorded, and the arithmetic mean of the number of initial defect seed points in all frames is calculated as the historical average seed point number, denoted by a sign. express.

[0038] For the current frame image to be processed, before generating the initial defect seed points, first use the current reference multiplier. As a preset multiple Perform preliminary seed point extraction, count the number of initial defect seed points detected in the current frame, and express them as symbols. This indicates the initial number of defect seed points in the current frame. Compared with the historical average number of seed points Compare them.

[0039] Set a preset ratio, using symbols express, The value can be either 1.5 or 2.0. Preset ratio. The setting is based on the principle that when the number of detected seed points deviates from the historical average by more than a certain multiple, it indicates a significant change in the anomaly level of the current frame, requiring adjustment of the discrimination sensitivity. If the initial number of defect seed points in the current frame... Greater than the historical average number of seed points preset ratio ,Right now Then the base multiple Increasing the step size by one value results in a preset increase by a multiple. The step size is indicated by a symbol... express, The value can be 0.1 or 0.2. The value of 0.1 is chosen to maintain a smooth change in the threshold while avoiding drastic fluctuations in sensitivity due to excessively large step sizes. The preset multiplier after increasing is... The original base multiple is replaced with the increased preset multiple. As a preset multiplier used for subsequent formal segmentation of the current frame. .

[0040] If the initial number of defect seed points in the current frame Number of seed points less than the historical average preset ratio The reciprocal of, that is Then the base multiple Decrease by one step value The preset reduction factor is obtained. The original base multiple is replaced with the reduced preset multiple. As a preset multiplier used for the formal segmentation of the current frame. .

[0041] If the initial number of defect seed points in the current frame In the range Within this range, the base multiple will be maintained. Preset multiples used for formal segmentation constant.

[0042] In the formal segmentation phase, the preset multiple determined after the above adaptive adjustment is used. Recalculate the first dynamic threshold Second dynamic threshold Based on this, the fused feature response map is re-thresholded, high outliers and low outliers are marked, and merged to obtain the final initial defect seed points, thus completing the extraction of the initial defect seed points for the current frame.

[0043] For the initial stage where the first batch or preliminary test results are insufficient, a standardized preset multiplier is used. The initial value is set to 3.0. This initial value is based on the fact that the probability of a deviation from the mean by more than three standard deviations under a standard normal distribution is less than 0.27%. Under the premise that the normal background follows an approximately normal distribution, this value can effectively separate background pixels from abnormal pixels. As the number of detection frames accumulates, the baseline multiplier... Based on historical data, the model gradually converges to a level that adapts to the surface characteristics of the current batch of plastic products.

[0044] In practice, an empty set of growth regions is established to store the connected defect regions that have completed growth. A grown marker matrix of the same size as the fused feature response map is initialized, and the values ​​of all elements in the grown marker matrix are set to the values ​​representing the ungrown state. An empty queue to be grown is established, and all initial defect seed points are added to the queue one by one. Simultaneously, the positions in the grown marker matrix corresponding to the initial defect seed points are marked with values ​​representing the grown state.

[0045] A pixel is taken from the queue to be grown and used as the current seed point. The upper, lower, left, and right adjacent pixels of the current seed point are searched. For each adjacent pixel, its state in the grown label matrix is ​​checked. If the adjacent pixel is already marked as grown, it is skipped and the next adjacent pixel is processed. If the adjacent pixel is not grown, the fused feature value of the adjacent pixel and the fused feature value of the current seed point are obtained. The absolute value of the difference between the fused feature value of the adjacent pixel and the fused feature value of the current seed point is calculated and compared with a preset similarity threshold. If the absolute value is less than the preset similarity threshold, the adjacent pixel is added to the current growth region, added to the queue to be grown, and marked as grown in the grown label matrix. If the absolute value is greater than or equal to the preset similarity threshold, the addition operation is not performed on the adjacent pixel.

[0046] The process of retrieving the next pixel from the queue to be grown as the current seed point, searching for the four neighboring pixels of the current seed point, and performing feature similarity judgment is repeated until the queue to be grown becomes empty. At this point, the growth process of a connected region ends, and the current grown region is stored as a connected defect region in the grown region set.

[0047] If there are still independent seed points in the initial defect seed point set that have not been included in any growth region, then select a new seed point from the unprocessed initial defect seed points, re-establish an empty growth region and an empty queue to be grown, add the new seed point to the queue to be grown and mark it as grown, repeat the above growth process to generate new connected defect regions until all initial defect seed points are included in a generated connected defect region.

[0048] The preset similarity threshold is dynamically determined before each region growth operation. The pixel values ​​of all non-seed point pixels in the fused feature response map are statistically analyzed; non-seed point pixels are all pixels in the fused feature response map that were not marked as initial defect seed points. The variance of all non-seed point pixel values ​​is calculated, and the variance is expressed as a percentage by the sign of the pixel. express, This is the expected value of the squared difference between the non-seed point pixel value and the mean of the non-seed point pixel values. The background fluctuation standard deviation is calculated, and the background fluctuation standard deviation is expressed as a sign... express, for The arithmetic square root of the background fluctuation standard deviation. Multiply by a preset scaling factor to obtain the dynamic similarity threshold. : in, For dynamic similarity threshold, This is a preset proportional coefficient. The standard deviation of background fluctuation. Preset scaling factor. The value range is set from 0.5 to 3.0, and is set to 1.5 during system initialization. The reason for setting it to 1.5 is that the standard deviation of background fluctuation is magnified to 1.5 times as the upper limit for similarity judgment, which can cover the fluctuation of fusion feature values ​​caused by texture changes within the same defect area, while avoiding growth crossing the boundary between the defect area and the normal background area. Dynamic similarity threshold Before generating each connected defect region, the non-seed point pixel distribution in the current frame fusion feature response map is recalculated once.

[0049] In specific implementation, please refer to Figure 4 For each connected defect region in the growth region set, boundary smoothing and hole filling operations are performed one by one. A connected defect region is selected from the growth region set, and its contour pixel sequence is extracted. The contour pixel sequence is defined as the set of all pixels adjacent to the external background pixels of the connected defect region, arranged in a clockwise or counterclockwise order. Morphological closing operations are used to iteratively dilate and erode the contour pixel sequence. The morphological closing operation uses a circular structuring element with a radius of 3 pixels. The reason for setting the radius to 3 pixels is that the contour burrs and depressions of common defects on the surface of plastic products are usually in the range of 2 to 5 pixels. A circular structuring element with a radius of 3 pixels can smooth small, irregular edges while preserving the macroscopic morphological features of the defect region. The number of iterations is set to 1 to 3. For each iteration, the binary mask image of the current connected defect region is first dilated using a circular structuring element. This dilation operation sets all pixels within the structuring element's coverage area as foreground pixels, filling in the depressions on the contour. Then, the result of the dilation operation is eroded using the same circular structuring element. This erosion operation shrinks the foreground pixels within the structuring element's coverage area back inward, eliminating burrs on the contour. After the iterations are complete, the connected defect region with smoothed boundaries is obtained.

[0050] The connected defect regions, after boundary smoothing, are converted into a binary mask image. Pixels covered by the connected defect regions in the binary mask image are assigned foreground identifier values, and the remaining regions are assigned background identifier values. Internal closed holes not covered by defect regions are detected in the binary mask image. Internal closed hole detection is achieved by finding connected background regions in the binary mask image completely surrounded by foreground identifier values. Connected component analysis is performed on the entire binary mask image, extracting all connected components composed of background identifier value pixels. Connected components touching the image boundary are excluded, and connected components completely surrounded by foreground identifier values ​​are retained. Each retained connected component corresponds to an internal closed hole.

[0051] In practice, for each detected closed hole, the set of boundary pixels of the closed hole is extracted, and then labeled with symbols. express, This is the set of coordinates of background pixels adjacent to the foreground identifier pixel within the closed hole. The set of boundary pixels of the defect region adjacent to the closed hole is extracted, and then labeled with a symbol... express, This is the set of coordinates of foreground pixels adjacent to the boundary pixels of closed holes within the current connected defect region. For the set of boundary pixels of closed holes... The coordinates of each pixel in Calculate the coordinates of the pixel and the set of pixels at the boundary of the defect region. The minimum Euclidean distance between the coordinates of all pixels in the array is expressed as a function of the Euclidean distance. Represents the minimum value of the Euclidean distance. The calculation method is to traverse... Coordinates of each pixel ,calculate and The Euclidean distance between the closed cavity boundary and the boundary of the adjacent defective region is taken as the minimum value among all calculation results. express, Take as Minimum Euclidean distance between all pixel coordinates The maximum value in: in, This is the Euclidean distance between the boundary of the closed cavity and the boundary of the adjacent defective region. The set of boundary pixels that close the hole. It is the set of boundary pixels of adjacent defective regions. and These represent the x and y coordinates of the pixels at the boundary of the closed hole, respectively. and These represent the x and y coordinates of the boundary pixels of adjacent defective regions, respectively. This indicates taking the maximum value among all calculated results. This means taking the minimum value among all calculated results. For closed hole boundary pixels pixels at the boundary of adjacent defect areas The Euclidean distance between them.

[0052] The calculated Euclidean distance Compare with a preset distance threshold. The preset distance threshold is expressed as a symbol. express, Set to 3 pixels. Preset distance threshold. The 3-pixel setting is based on the fact that if the distance between a cavity inside a defective area on the surface of a plastic product and the boundary of the defective area exceeds 3 pixels, the cavity may belong to the normal surface texture rather than being part of the defective connected region. Only when the cavity is adjacent to the boundary of the defective area is it considered an internal depression of the defect. If the Euclidean distance... Less than the preset distance threshold Then, all pixels within the closed hole will be filled as part of the defect area, and the pixels belonging to the closed hole will be modified from background identifier values ​​to foreground identifier values ​​in the binary mask image. If the Euclidean distance... Greater than or equal to the preset distance threshold If the closed hole is not filled, the closed hole area will remain as the background identifier value.

[0053] In practice, after performing boundary smoothing and hole filling operations on each connected defect region in the growth region set, all processed connected defect regions are merged and output. The merging method involves performing a logical OR operation on the binary mask images of each processed connected defect region to obtain the binary mask result for full-image defect detection. In this full-image defect detection binary mask result, the pixel position corresponding to the foreground identifier value is the region identified as a defect in the final defect detection result.

[0054] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A machine vision-based online defect detection method for plastic products, characterized in that, Includes the following steps: Acquire continuous multi-frame surface images of plastic products, and calculate local binary pattern texture features and histogram edge features of orientation gradient for each frame; The local binary pattern texture features and the directional gradient histogram edge features are weighted and fused according to pixel position to generate a fused feature response map; Adaptive threshold segmentation is performed on the fused feature response map to obtain initial defect seed points; Using the initial defect seed point as the starting point for growth, region growth is performed using four-neighbor connectivity and feature similarity constraints to obtain connected defect regions. Perform boundary smoothing and hole filling on the connected defective regions, and output the final defect detection results.

2. The online defect detection method for plastic products based on machine vision according to claim 1, characterized in that, The specific steps for acquiring continuous multi-frame surface images of the plastic product are as follows: Multiple industrial cameras are used to simultaneously capture images of the surface of plastic products from different angles, and each industrial camera outputs an image sequence at a fixed frame rate. The images acquired by all cameras at the same time are subjected to a viewpoint normalization transformation based on calibration parameters, and the images from different viewpoints are projected onto a unified spatial reference plane. On the unified spatial reference plane, phase correlation matching is performed on the normalized images at adjacent time points to compensate for the positional offset of the plastic product on the conveyor belt, thereby obtaining aligned continuous multi-frame surface images.

3. The online defect detection method for plastic products based on machine vision according to claim 2, characterized in that, The phase correlation matching uses Fourier transform to calculate the translation amount between adjacent frame images, and performs reverse translation compensation on the subsequent frame image based on the translation amount.

4. The online defect detection method for plastic products based on machine vision according to claim 1, characterized in that, The specific steps for generating a fused feature response map by weighting and fusing the local binary pattern texture features and the oriented gradient histogram edge features according to pixel position are as follows: For each pixel in each frame of the image, calculate the gray-level difference of the sampling points on its neighborhood ring, generate a local binary pattern code, and statistically analyze the histogram distribution of the code as the local binary pattern texture feature value of the pixel. The gradient direction within the local window containing the pixel is quantized and voted to construct an oriented gradient histogram. The magnitude of the main gradient direction is taken as the edge feature value of the oriented gradient histogram of the pixel. The local binary pattern texture feature value is multiplied by a first weighting coefficient, and the directional gradient histogram edge feature value is multiplied by a second weighting coefficient. The two are added together to obtain the fusion feature value of the pixel. The fusion feature response map is generated by traversing all pixels in the image.

5. The online defect detection method for plastic products based on machine vision according to claim 1, characterized in that, The specific steps for performing adaptive threshold segmentation on the fused feature response map to obtain initial defect seed points are as follows: The distribution of all pixel values ​​in the fused feature response map is statistically analyzed, and the mean and standard deviation of the entire response map are calculated. The first dynamic threshold is obtained by adding the mean of the entire image to a preset multiple of the standard deviation, and the second dynamic threshold is obtained by subtracting the preset multiple of the standard deviation from the mean of the entire image. Pixels with pixel values ​​greater than the first dynamic threshold in the fused feature response map are marked as high anomalies, and pixels with pixel values ​​less than the second dynamic threshold are marked as low anomalies. The high anomaly points and the low anomaly points are merged as the initial defect seed points.

6. The online defect detection method for plastic products based on machine vision according to claim 1, characterized in that, Using the initial defect seed point as the starting point for growth, the specific steps for obtaining connected defect regions through region growth using four-neighbor connectivity and feature similarity constraints are as follows: Establish an empty set of growth regions and add all initial defect seed points to the growth queue; Take a seed point from the queue to be grown, and search for the four adjacent pixels above, below, left, and right of the seed point; For each adjacent pixel, calculate the absolute difference between the fusion feature value of the adjacent pixel and the fusion feature value of the current seed point. If the absolute difference is less than the preset similarity threshold and the adjacent pixel has not been marked as grown, then add the adjacent pixel to the growth region and add it to the queue to be grown. Repeat the above extraction and search steps until the growth queue is empty, and then store the current growth region into the connected defect region set.

7. The online defect detection method for plastic products based on machine vision according to claim 6, characterized in that, The preset similarity threshold is dynamically determined based on the texture fluctuation range of the background area on the surface of the plastic product: The variance of all non-seed point pixel values ​​in the fused feature response map is calculated, and the square root of the variance is used as the standard deviation of background fluctuation. The dynamic similarity threshold is obtained by multiplying the standard deviation of the background fluctuation by a preset scaling factor.

8. The online defect detection method for plastic products based on machine vision according to claim 1, characterized in that, The specific steps for performing boundary smoothing and hole filling on the connected defective regions and outputting the final defect detection result are as follows: For each connected defective region, its contour pixel sequence is extracted, and morphological closing operation is used to perform iterative dilation and erosion operations on the contour pixel sequence to eliminate burrs and depressions on the contour. The connected defective regions that have undergone boundary smoothing are converted into binary mask images, and internal closed voids not covered by the defective regions are detected in the binary mask images. For each detected closed hole, calculate the Euclidean distance between the hole boundary and the boundary of the adjacent defect area. If the Euclidean distance is less than a preset distance threshold, then fill the hole pixel as part of the defect area. All connected defect areas after hole filling are merged and output as the final defect detection result.

9. The online defect detection method for plastic products based on machine vision according to claim 4, characterized in that, The first weighting coefficient and the second weighting coefficient are dynamically adjusted according to the surface material type of the plastic product: When the surface of the plastic product is a high-gloss material, the first weighting coefficient is set to a first value, and the second weighting coefficient is set to a second value, wherein the first value is greater than the second value; When the surface of the plastic product is a frosted material, the first weighting coefficient is set to the third value, and the second weighting coefficient is set to the fourth value, wherein the third value is less than the fourth value; When the surface of the plastic product is transparent, the first weighting coefficient is set to the fifth value, the second weighting coefficient is set to the sixth value, and the ratio of the fifth value to the sixth value is equal to the reciprocal of the haze value of the transparent material.

10. The online defect detection method for plastic products based on machine vision according to claim 5, characterized in that, The preset multiple of the standard deviation is adaptively adjusted based on the frequency of defects in plastic products across multiple consecutive frames of images: Analyze the pre-test results of the current batch of plastic products and calculate the historical average of the deviation factor of the defective area from the mean in the fused feature response map. Using the historical average as a benchmark multiple, if the number of initial defect seed points detected in the current frame is greater than a preset ratio of the historical average number of seed points, the benchmark multiple is increased by a step value to obtain an increased preset multiple. If the number of initial defect seed points detected in the current frame is less than a preset proportion of the historical average number of seed points, then the reference multiple is reduced by one step value to obtain a reduced preset multiple.

Images