A mold defect visual inspection method

Abstract

The application discloses a kind of mould defect visual detection methods, specifically related to industrial machine vision detection technical field, for solving the technical problem that image appears spatial variation blur due to mechanical vibration under mobile shooting condition;It is by obtaining the image to be detected on the surface of mould, the gradient amplitude of each region is analyzed, the blur characteristics of different regions in the image to be detected are determined according to the difference of gradient amplitude, the expected confidence of each region based on blur characteristics is evaluated to execute deblurring operation, the preliminary restoration image is obtained by executing deblurring operation to region, the artifact index is calculated in the uniform background area of preliminary restoration image, the distribution concentration degree of image composition in frequency domain in each local feature area in preliminary restoration image is analyzed, the distribution concentration degree is compared with the preset defect judgment threshold adjusted according to artifact index, whether corresponding local feature area is defect area is judged;Realize accurate identification to mould surface defect under complex imaging condition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial machine vision detection, and in particular to a mold defect visual detection method. BACKGROUND

[0002] In the field of industrial manufacturing, the health status of the mold directly determines the quality of the final product. In order to realize real-time monitoring and predictive maintenance of the mold state in the production process, the online automatic detection method based on machine vision has become the development trend of the industry. This method usually takes image acquisition equipment mounted on a mechanical arm or a mobile platform to take images of the mold cavity surface within the production line beat, and uses image processing and analysis algorithms to identify various defects. The effectiveness of such a method is highly dependent on the representation quality of the collected images.

[0003] However, when implementing online detection, the image acquisition unit is in a continuous motion state. The high dynamic start-stop and positioning process of the image acquisition unit will introduce wideband mechanical vibration, which is transmitted to the image sensor, resulting in unpredictable and non-uniform relative micro-movement between the sensor and the mold surface target at the exposure moment. What is particularly critical is that the high-contrast micro-texture (such as machining tool marks) inherent to the mold cavity surface interacts with the above-mentioned complex micro-movement, making the generated image degradation model (i.e., the point spread function) exhibit significant spatial heterogeneity, which severely damages the integrity of key discriminative features such as edges and contours of the defect target, resulting in a significant reduction in the detection rate and classification accuracy of the subsequent recognition algorithm for small defects, which cannot meet the stringent requirements of industrial detection for reliability. SUMMARY

[0004] The present application provides a mold defect visual detection method to solve the technical problems in the prior art.

[0005] The technical solution of the present application to solve the above technical problems is as follows:

[0006] A mold defect visual detection method, comprising:

[0007] S1, acquiring a to-be-detected image of a mold surface by an image acquisition device mounted on a mobile platform;

[0008] S2, analyzing the gradient amplitude of each region in the to-be-detected image, and determining the blur characteristics of different regions in the to-be-detected image according to the difference in the gradient amplitude;

[0009] S3, evaluating the expected confidence of performing a deblurring operation based on the blur characteristics of each region, and performing the deblurring operation on each region based on the expected confidence to obtain a preliminary restored image;

[0010] S4, calculating information entropy values of a plurality of local windows in different directions in a uniform background area of the preliminary restored image and calculating an anisotropy ratio of local entropy based on the information entropy values, quantifying the anisotropy ratio as a pseudo-index;

[0011] S5, demarcating a plurality of local feature areas in the preliminary restored image and analyzing a distribution concentration degree of image components in a frequency domain in each local feature area;

[0012] S6, comparing the distribution concentration degree with a preset defect judgment threshold adjusted according to the pseudo-index, and determining whether the corresponding local feature area is a defect area.

[0013] Further, the image acquisition device carried by the mobile platform is used to acquire the to-be-detected image of the mold surface, comprising:

[0014] The mobile platform is controlled to move to a detection point on the mold surface along a preset scanning path;

[0015] When the mobile platform stably stays at the detection point, the image acquisition device is triggered to acquire the to-be-detected image of the mold surface with predetermined exposure parameters.

[0016] Further, the process of acquiring the to-be-detected image comprises controlling the light source to irradiate the mold surface with constant illumination and angle.

[0017] Further, the gradient amplitude of each region in the to-be-detected image is analyzed, and the blur characteristics of different regions in the to-be-detected image are determined according to the difference in gradient amplitude, comprising:

[0018] The Sobel operator is used to calculate the horizontal direction gradient value and the vertical direction gradient value of each pixel point in the to-be-detected image;

[0019] The gradient amplitude of the corresponding pixel point is calculated according to the horizontal direction gradient value and the vertical direction gradient value of each pixel point;

[0020] The to-be-detected image is divided into a plurality of rectangular regions with the same size, and the average value of the gradient amplitude of all pixel points in each rectangular region is calculated;

[0021] According to the difference in the average value of the gradient amplitude of the plurality of rectangular regions, the blur characteristics of different regions in the to-be-detected image are determined.

[0022] Further, according to the difference in the average value of the gradient amplitude of the plurality of rectangular regions, the blur characteristics of different regions in the to-be-detected image are determined, comprising:

[0023] The region with the average value of the gradient amplitude lower than the first preset threshold is determined as a high blur region, the region with the average value of the gradient amplitude between the first preset threshold and the second preset threshold is determined as a medium blur region, and the region with the average value of the gradient amplitude higher than the second preset threshold is determined as a low blur region.

[0024] Further, an expected confidence of performing a deblurring operation on each region based on the blur characteristic is evaluated, and a preliminary restored image is obtained by performing the deblurring operation on each region based on the expected confidence, including:

[0025] For each region for which the blur characteristic has been determined, a structure tensor thereof at a plurality of different integral scales is calculated;

[0026] Eigenvalues of the structure tensor at each integral scale are decomposed, and a principal eigenvalue of each structure tensor is extracted;

[0027] The principal eigenvalues at all integral scales are arranged in a scale order to form an eigenvalue trajectory;

[0028] A standard deviation of the eigenvalue trajectory is calculated as a quantitative indicator of trajectory stability;

[0029] The quantitative indicator of trajectory stability is compared with a stability threshold, and a region with the quantitative indicator of trajectory stability less than or equal to the stability threshold is assigned a high confidence rating, and a region with the quantitative indicator of trajectory stability higher than the stability threshold is assigned a low confidence rating;

[0030] A standard deblurring operation is performed on the region with the high confidence rating, and a weakened deblurring operation is performed on the region with the low confidence rating, and finally a preliminary restored image is obtained.

[0031] Further, the Lucy-Richardson deconvolution algorithm is used to perform the standard deblurring operation on the region with the high confidence rating, and the Lucy-Richardson deconvolution algorithm with reduced iteration times is used to perform the weakened deblurring operation on the region with the low confidence rating.

[0032] Further, in a uniform background region of the preliminary restored image, information entropy values of a plurality of local windows in different directions are calculated, and an anisotropy ratio of local entropy is calculated based on the information entropy values, and the anisotropy ratio is quantified as an artifact indicator, including:

[0033] A region with a gray variance lower than a variance threshold is selected as the uniform background region in the preliminary restored image;

[0034] A plurality of local windows are arranged in a sliding manner in the uniform background region;

[0035] For each local window, a gray co-occurrence matrix thereof in four directions of zero degrees, forty-five degrees, ninety degrees, and one hundred thirty-five degrees is calculated.

[0036] calculating the information entropy value of the corresponding direction based on the gray level co-occurrence matrix in each direction;

[0037] For each local window, the standard deviation of the information entropy values of the four directions is calculated;

[0038] The average of the standard deviations of all local windows is quantified as an artifact index.

[0039] Further, a plurality of local feature regions are demarcated in the preliminary restored image, and the distribution concentration degree of image components in the frequency domain in each local feature region is analyzed, including:

[0040] The preliminary restored image is divided into a plurality of non-overlapping rectangular regions as local feature regions;

[0041] The image gray scale values in each local feature region are subjected to two-dimensional fast Fourier transform to obtain the corresponding frequency domain spectrum;

[0042] The energy distribution of each frequency domain spectrum is calculated;

[0043] The energy distribution entropy value of the corresponding frequency domain spectrum is calculated based on the energy distribution of each frequency domain spectrum;

[0044] The calculated energy distribution entropy value is taken as the quantification index of the distribution concentration degree of image components in the frequency domain in the corresponding local feature region.

[0045] Further, the distribution concentration degree is compared with a preset defect judgment threshold adjusted according to the artifact index to determine whether the corresponding local feature region is a defect region, including:

[0046] The energy distribution entropy value corresponding to each local feature region is obtained as the quantification index of the distribution concentration degree;

[0047] The artifact index is obtained;

[0048] The artifact index is multiplied by a normalization weight coefficient to obtain a normalized artifact influence factor;

[0049] The preset defect judgment threshold is added to the normalized artifact influence factor to obtain an adjusted judgment threshold;

[0050] The energy distribution entropy value of each local feature region is compared with the adjusted judgment threshold;

[0051] When the energy distribution entropy value of the local feature region is lower than the adjusted judgment threshold, the corresponding local feature region is determined to be a defect region.

[0052] The beneficial effects of the present application are:

[0053] 1. By analyzing the gradient amplitude difference of each region in the image to be detected, the spatially varying blur characteristics are accurately identified, effectively solving the non-uniform blur problem caused by mechanical vibration; based on the blur characteristics, the expected confidence of the deblurring operation is evaluated, realizing the differential processing of different regions, which can effectively restore the edge and contour features of defects, avoid the loss of details caused by over-processing, significantly improve the integrity of the key discriminative information in the image, and provide a high-quality image basis for subsequent accurate detection.

[0054] 2. By quantifying the artifact index generated in the deblurring process, a scientific artifact evaluation system is established, and based on the index, the defect judgment threshold is dynamically adjusted, which can effectively distinguish between real defects and artifact features introduced by the algorithm, while maintaining the sensitivity of micro-defect detection, significantly reducing the risk of misjudgment, effectively overcoming the limitations of traditional fixed threshold method, significantly improving the reliability and accuracy of the detection result, fully meeting the dual needs of stability and accuracy in industrial detection. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 A flowchart of the mold defect visual detection method of the present application.

[0056] Figure 2 A flowchart of generating a preliminary restored image of the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0058] As shown in Figure 1 The present application provides a mold defect visual detection method, comprising:

[0059] S1, acquiring the image to be detected on the surface of the mold by the image acquisition device carried by the moving platform;

[0060] S2, analyzing the gradient amplitude of each region in the image to be detected, and determining the blur characteristics of different regions in the image to be detected according to the difference of the gradient amplitude;

[0061] S3, evaluating the expected confidence of the deblurring operation of each region based on the blur characteristics, and performing the deblurring operation on each region based on the expected confidence to obtain a preliminary restored image;

[0062] S4. Calculate the information entropy values of a plurality of local windows in different directions in the uniform background area of the preliminary restored image, and calculate the anisotropy ratio of local entropy based on the information entropy values, and quantize the anisotropy ratio as an artifact index;

[0063] S5. Delimit a plurality of local feature areas in the preliminary restored image, and analyze the distribution concentration degree of image components in the frequency domain in each local feature area;

[0064] S6. Compare the distribution concentration degree with a preset defect judgment threshold adjusted according to the artifact index, and determine whether the corresponding local feature area is a defect area.

[0065] S1. Obtain a to-be-detected image of a mold surface by an image acquisition device carried by a mobile platform, and the specific implementation includes:

[0066] First, generate a preset scanning path of the mobile platform according to a three-dimensional model of the to-be-detected mold and a pre-set detection planning scheme; the preset scanning path contains a series of detection site spatial coordinates arranged in sequence, ensuring complete coverage of all to-be-detected surface areas of the mold; the mobile platform is driven by a high-precision servo motor, and moves accurately along the preset scanning path according to position instructions issued by a control system; when the spatial three-dimensional deviation between the optical center point of the image acquisition device carried by the mobile platform and the target detection site coordinate is less than the positioning tolerance allowed by the system, for example, less than 10 microns, it is determined that the mobile platform has accurately moved to the detection site.

[0067] After the mobile platform stably resides at the detection site, image acquisition is not triggered immediately, but enters a short stable waiting period, for example, the waiting time is set to 100 milliseconds; during this period, the system continuously monitors the residual vibration amplitude of the platform through the high-sensitivity vibration sensor carried by the platform; when the monitored vibration amplitude continuously falls below the preset stable threshold, for example, continuously below 0.5 microns per second for 50 milliseconds, it is determined that the mobile platform has reached a stable state, and then the control system sends a trigger acquisition signal to the image acquisition device.

[0068] The image acquisition device calls a pre-stored predetermined exposure parameter set to perform image acquisition after receiving the trigger signal; the predetermined exposure parameter set is determined by a systematic calibration experiment in advance, and a specific setting process thereof includes: under the condition of strictly same standard light source illumination, taking a standard test block of a typical material of a mold surface as an object, performing a plurality of groups of shooting experiments with different exposure times, aperture sizes and gain values; by analyzing the gray level histogram statistical characteristics of the acquired images, selecting the group of parameters that can make the overall gray level mean value of the image in the middle section of the camera dynamic range, for example, in the interval of 40% to 60%, and the maximum gray level standard deviation as the final predetermined exposure parameter, for example, specifically setting the exposure time to 20 milliseconds, the aperture value to F8, and the gain value to 1.0; the acquisition device controls the optical lens and the sensor to complete the acquisition and digital conversion of a single to-be-detected image according to the group of parameters.

[0069] During the entire image acquisition process, the illumination condition of the light source is strictly controlled; a high-illuminance ring LED white light source is adopted, and the power supply is driven by a high-precision constant current source to ensure the high stability of the luminous intensity; the illuminance value of the light source is measured at a typical position on the mold surface by a calibrated illuminometer and is fed back to the light source controller in real time, and the illuminance is stabilized at a preset value, for example, 1000 lux, by a closed-loop control algorithm, and the fluctuation range is controlled within ±3%; at the same time, the light source is firmly fixed by a high-rigidity mechanical support, so that the light emitting plane of the light source and the normal direction of the mold surface form a fixed included angle, for example, 45 degrees; the angle is calibrated by a high-precision digital angle gauge during system installation and debugging and is mechanically locked to ensure that the illumination angle and the illuminance of each detection site remain constant and consistent during the entire scanning acquisition process, thereby providing consistent and reliable input for subsequent image processing and analysis.

[0070] S2, analyzing the gradient amplitude of each region in the to-be-detected image, and determining the blur characteristics of different regions in the to-be-detected image according to the difference in the gradient amplitude, including:

[0071] After obtaining the to-be-detected image, the image blur characteristic analysis process is performed. First, the Sobel operator is used to calculate the gradient component of each pixel point in the to-be-detected image; in specific implementation, a 3-pixel x 3-pixel horizontal direction convolution kernel is used for convolution operation on the image, the numerical configuration of the convolution kernel is that the left column is -1, the middle column is 0, and the right column is 1, and the gradient value of each pixel point in the horizontal direction is obtained through the convolution operation; at the same time, a 3-pixel x 3-pixel vertical direction convolution kernel is used for convolution operation, the numerical configuration of the convolution kernel is that the upper row is -1, the middle row is 0, and the lower row is 1, and the gradient value of each pixel point in the vertical direction is obtained through the convolution operation; the gradient values in the two directions are signed scalars, and the numerical size reflects the degree of gray level change of the pixel point in the direction.

[0072] After obtaining the horizontal gradient value and the vertical gradient value of each pixel, the gradient amplitude of the pixel is obtained by calculating the square root of the sum of squares of the two gradient values; the specific calculation process is: multiplying the horizontal gradient value by itself, multiplying the vertical gradient value by itself, adding the two product results, and then performing square root operation on the addition result; this calculation process is executed for each pixel in the image one by one, and finally a gradient amplitude image with the same size as the original image to be detected is obtained, each pixel value in the image represents the gradient amplitude of the corresponding position of the original image, and the value is a non-negative scalar, and the larger the value, the more obvious the image edge or texture feature of the point.

[0073] After completing the gradient amplitude calculation of the whole image, the entire image to be detected is divided into multiple rectangular regions with the same size; the determination of the size of the rectangular region needs to balance the analysis accuracy and the calculation efficiency, for example, 32 pixels x 32 pixels can be selected as the size of a rectangular region; when dividing, start from the first pixel in the upper left corner of the image, move to the right and down with a fixed step, ensure that each rectangular region neither overlaps nor completely covers the whole image; for the part of the image right edge and lower edge that cannot be covered by the complete rectangular region, the mirror filling method is used to supplement the pixels, that is, the method of copying the pixel value of the image edge is used to form a complete rectangular region.

[0074] For each rectangular region obtained by dividing, the arithmetic mean of the gradient amplitudes of all the pixels contained in the rectangular region is calculated; the average value represents the average edge strength or texture richness of the image content in the rectangular region; when calculating, the gradient amplitudes of each pixel in the region are added, and then divided by the total number of pixels in the region to obtain the average value of the gradient amplitude; this calculation process is executed for all rectangular regions to obtain a list containing the average values of the gradient amplitudes of all rectangular regions.

[0075] After obtaining the average values of the gradient amplitudes of all rectangular regions, the first preset threshold and the second preset threshold for distinguishing the blur degree are set according to the statistical distribution characteristics of these values; the specific setting method is: collecting the average values of the gradient amplitudes of all rectangular regions, arranging these values in order from small to large, calculating the cumulative percentage of each value in the sorted sequence, taking the average value of the gradient amplitude at the cumulative percentage of 25% as the first preset threshold, and taking the average value of the gradient amplitude at the cumulative percentage of 75% as the second preset threshold; this threshold determination method based on statistical quantile can adapt to the contrast characteristics of different images, and ensure that the threshold setting matches the image content.

[0076] Finally, according to the size relationship between the average value of the gradient amplitude of each rectangular region and the two threshold values, the blur characteristics of the rectangular region are classified; the rectangular region with the average value of the gradient amplitude lower than the first preset threshold value is marked as a high blur region, and such a region usually corresponds to a flat surface in the image lacking texture features; the rectangular region with the average value of the gradient amplitude greater than or equal to the first preset threshold value and less than or equal to the second preset threshold value is marked as a moderate blur region, and such a region usually corresponds to a transition region in the image having certain texture but not clear; the rectangular region with the average value of the gradient amplitude higher than the second preset threshold value is marked as a low blur region, and such a region usually corresponds to a feature region in the image with clear edges and obvious texture; through this classification mode, different regions in the image to be detected are quantitatively divided according to the blur characteristics, thereby providing a basis for subsequent targeted deblurring processing; the entire analysis process is completely based on the gradient features of the image itself, and the blur characteristic determination of all regions can be automatically completed without manual intervention.

[0077] Figure 2 A flowchart of generating the preliminary restored image is given, the step S3 of evaluating the expected confidence of performing the deblurring operation based on the blur characteristics of each region, and the step of performing the deblurring operation on each region based on the expected confidence to obtain the preliminary restored image, and the specific implementation includes:

[0078] After the determination of the blur characteristics of each region in the image to be detected, the evaluation of the expected confidence of performing the deblurring operation and the differential processing process based on the evaluation result are performed. For each region with the determined blur characteristics, the structure tensor under multiple different integral scales needs to be calculated first, and the selection of the integral scale is based on the specific circumstances of the region size and the blur degree, for example, for a 32 pixel x 32 pixel region, 5 different integral scales can be selected, and these integral scales are 3 pixel x 3 pixel, 5 pixel x 5 pixel, 7 pixel x 7 pixel, 9 pixel x 9 pixel and 11 pixel x 11 pixel; the calculation of the structure tensor is realized through the following mode: first, the horizontal direction gradient value and the vertical direction gradient value of each pixel point in the region are calculated, and these gradient values are obtained through the Sobel operator calculation; then, the horizontal direction gradient value of each pixel point is multiplied by itself to obtain the horizontal direction gradient square value of the pixel point, and the horizontal direction gradient value of each pixel point is multiplied by the vertical direction gradient value to obtain the mixed gradient product value; the vertical direction gradient value of each pixel point is multiplied by itself to obtain the vertical direction gradient square value of the pixel point; finally, the three calculation values are respectively subjected to Gaussian weighted summation within the selected integral scale window, and the standard deviation of the Gaussian weighting is set to one sixth of the integral scale size, and through this mode, the structure tensor matrix under each scale is obtained, and the matrix is a 2 x 2 symmetric matrix.

[0079] The structural tensor matrix calculated at each integral scale is subjected to eigenvalue decomposition, which is realized by solving the characteristic equation, i.e. calculating the eigenroot when the determinant of the matrix is zero. The specific calculation process is as follows: for the four elements of the structural tensor matrix, denoted as a, b, c and d respectively, where a is the Gaussian weighted sum of the horizontal gradient square, b and c are the Gaussian weighted sums of the mixed gradient product values, and d is the Gaussian weighted sum of the vertical gradient square, the eigenroot of the characteristic equation is solved, and the larger one of the two eigenvalues is taken as the principal eigenvalue at the scale. The principal eigenvalue reflects the main intensity information of the image structure at the scale, and its numerical value is positively correlated with the image texture definition at the scale.

[0080] The principal eigenvalues at all integral scales are arranged in ascending order of scale to form a numerical sequence, which is called the eigenvalue trajectory. The trajectory describes the regularity of the image structure intensity change with the observation scale. The standard deviation of the eigenvalue trajectory is calculated as a quantitative indicator of the stability of the trajectory. The calculation process is as follows: first, calculate the arithmetic mean of all principal eigenvalues, then square the difference between each principal eigenvalue and the arithmetic mean, calculate the arithmetic mean of these squared values, and finally take the square root of the arithmetic mean to obtain the quantitative indicator of the stability of the trajectory. The smaller the indicator value, the more stable the eigenvalue trajectory.

[0081] The setting of the stability threshold is based on the statistical analysis results of a large number of sample images. For example, 1000 mold surface images with different degrees of blur can be collected, the standard deviation values of the eigenvalue trajectories of each region are calculated, and these standard deviation values are arranged in ascending order. The value at the 85th percentile of the arrangement order is taken as the stability threshold. The stability threshold obtained by this method is usually between 0.15 and 0.25, and the specific value depends on the image acquisition conditions and the characteristics of the mold surface. The quantitative indicator of the stability of each region is compared with the stability threshold. The region with a quantitative indicator of the stability less than or equal to the stability threshold is assigned a high confidence rating, indicating that the structural features of the region are stable at different scales and are suitable for strong deblurring processing. The region with a quantitative indicator of the stability higher than the stability threshold is assigned a low confidence rating, indicating that the structural features of the region change greatly with the scale and need to use a more conservative deblurring processing strategy.

[0082] For the regions with high confidence rating, a standard deblurring operation is performed using the Lucy-Richardson deconvolution algorithm with 50 iterations, a 7x7 pixel kernel size, and a Gaussian kernel shape with a standard deviation of 2.0. For the regions with low confidence rating, a weakened deblurring operation is performed using the Lucy-Richardson deconvolution algorithm with 10 iterations, a 7x7 pixel kernel size, and a Gaussian kernel shape with a standard deviation of 3.0 to reduce the deblurring intensity. After processing all regions, the results are recombined according to the original positional relationships to obtain a complete preliminary restoration image. This image maintains the details of clear regions while avoiding over-processing in blurred regions, providing a better quality input image for subsequent defect detection and analysis. The entire process is fully adaptive to image content characteristics, requiring no manual intervention for differential processing of different regions.

[0083] Step S3 evaluates the deblurring confidence by analyzing the stability of the eigenvalue trajectories of the multi-scale structure tensor. Regions where the structure features remain stable at different observation scales are considered to have reliable image structures and are suitable for strong deblurring. Regions with large eigenvalue fluctuations may contain noise or inherent blur, and blind deblurring may amplify distortion. This approach discards the conventional uniform deblurring strategy for all regions and instead makes adaptive decisions based on the structural stability of the image content, achieving a better balance between enhancing texture details and suppressing artifact noise. This effectively addresses the common problem of secondary damage caused by excessive deblurring in existing technologies.

[0084] S4, in the uniform background region of the preliminary restoration image, calculate the information entropy values of multiple local windows in different directions and calculate the anisotropy ratio of local entropy based on the information entropy values, and quantify the anisotropy ratio as an artifact index, including:

[0085] After obtaining the preliminary restoration image, the artifact index quantification process is performed. First, select a region with a low gray variance from the preliminary restoration image as a uniform background region, and the determination method of the variance threshold is as follows: calculate the gray variance value of all possible regions in the preliminary restoration image, arrange these variance values in order from small to large, and take the value at the 10th percentile position as the variance threshold. The variance threshold obtained in this way can ensure that the selected region is uniform enough. For example, for an 8-bit grayscale image, the threshold is usually between 15 and 25 gray levels squared, and the specific value depends on the actual content characteristics of the image.

[0086] After determining the uniform background regions, a plurality of local windows are arranged in these regions in a sliding manner. The size of the local window needs to be selected by balancing the calculation accuracy and the feature expression capability. For example, 16 pixels x 16 pixels can be selected as the size of the local window. The sliding step is set based on the window size, and is usually set to be half of the window size. For example, when the window size is 16 pixels x 16 pixels, the sliding step is set to be 8 pixels. When arranging, the first pixel in the top-left corner of the uniform background region is taken as a starting point, and is moved to the right and downward by a fixed sliding step to ensure that the entire uniform background region is covered. For the positions where the edge of the region cannot accommodate a complete window, the pixels are supplemented in a mirror filling manner.

[0087] For each local window, a gray level co-occurrence matrix in four directions of zero degrees, forty-five degrees, ninety degrees and one hundred thirty-five degrees is calculated. When calculating, a distance parameter between pixel pairs is first determined, which is usually set to be 1 pixel. For the zero-degree direction, the gray level value appearance frequency of all horizontally adjacent pixel pairs in the window is counted, i.e., the gray level combination of each pixel and its right adjacent pixel. For the forty-five-degree direction, the gray level value appearance frequency of all right-up to left-down adjacent pixel pairs in the window is counted, i.e., the gray level combination of each pixel and its right-up adjacent pixel. For the ninety-degree direction, the gray level value appearance frequency of all vertically adjacent pixel pairs in the window is counted, i.e., the gray level combination of each pixel and its lower adjacent pixel. For the one hundred thirty-five-degree direction, the gray level value appearance frequency of all left-up to right-down adjacent pixel pairs in the window is counted, i.e., the gray level combination of each pixel and its left-down adjacent pixel. The gray level co-occurrence matrix in each direction is a two-dimensional 256 x 256 matrix, wherein the value of each element represents the frequency of the corresponding gray level value combination appearing in the specified direction and distance, and the row index and column index of the matrix correspond to the gray level values of two pixels, respectively.

[0088] Based on the gray level co-occurrence matrix in each direction, the information entropy value in the corresponding direction is calculated. The calculation process of the information entropy value is as follows: first, each element value in the gray level co-occurrence matrix is divided by the sum of all element values in the matrix to obtain a normalized probability distribution matrix; then, the negative value of the probability value multiplied by the logarithmic probability value with a base of 2 is calculated for each probability value in the matrix; and finally, the sum of all calculation results is obtained. This calculation process is performed on the gray level co-occurrence matrices in the four directions respectively to obtain the information entropy values in the four directions. These entropy values reflect the complexity and randomness of the image texture in different directions. The greater the entropy value, the more complex the texture.

[0089] For each local window, the standard deviation of its four directional information entropy values is calculated; the calculation process of the standard deviation is as follows: first, the arithmetic mean of the four information entropy values is calculated, then the difference between each information entropy value and the arithmetic mean is squared, the arithmetic mean of these square values is calculated, and finally the square root of the arithmetic mean is taken; the obtained standard deviation value reflects the anisotropy degree of the texture direction distribution in the local window, the larger the value, the stronger the anisotropy, and the smaller the value, the better the isotropy.

[0090] The arithmetic mean of the standard deviation values of all local windows is calculated, and this average value is quantified as an artifact index; when calculating the arithmetic mean, the standard deviation values of all local windows are added and then divided by the total number of local windows; the final obtained artifact index is a dimensionless value, and its size reflects the severity of the artificial artifact introduced by the deblurring operation in the preliminary restored image, the larger the value, the more serious the artifact, and the smaller the value, the better the image quality; this artifact index will be used for defect judgment threshold adjustment in the subsequent steps to ensure that the accuracy of defect detection is not affected by the deblurring artifact. The entire calculation process is completely based on the statistical characteristics of the image itself, without the need for external parameter input, and realizes self-adaptive quantitative evaluation.

[0091] Step S4 proposes an artifact quantification index based on the anisotropy ratio of texture in a uniform background area. Existing evaluation methods mostly focus on global clarity improvement, often ignoring the side effect that the deblurring algorithm may introduce directional artificial texture (artifact) in the originally uniform background. By calculating the dispersion degree of texture entropy values in multiple directions to quantify this undesirable directional preference, the subjective feeling of artifact is converted into an objective measure. This enables the subsequent defect judgment to actively avoid "false features" introduced by the algorithm itself, significantly improving the accuracy and reliability of defect recognition, and solving the technical bias of misjudging artifacts as real defects.

[0092] S5, multiple local feature regions are delineated in the preliminary restored image, and the distribution concentration degree of image components in the frequency domain in each local feature region is analyzed, including:

[0093] After obtaining the preliminary restoration image, a process of analyzing the concentration of frequency domain distribution is performed. First, the preliminary restoration image is divided into a plurality of non-overlapping rectangular regions as local feature regions, the size of the rectangular region is selected based on the size range of typical defects of the mold surface and the resolution characteristics of the image, for example, 32 pixels x 32 pixels can be selected as the size of a local feature region, and this size is determined by analyzing the size of the smallest detectable defect in historical defect data, and taking 2 to 3 times of the size as the size of the local feature region; when dividing, starting from the first pixel in the upper left corner of the image, dividing to the right and down with the selected size as the fixed step size, ensuring that each local feature region is completely independent and covers the entire image, for the remaining part of the image right edge and lower edge that cannot form a complete rectangular region, the mirror image expansion method is used to supplement the pixels to form a complete local feature region, and the mirror image expansion method is to copy the pixel values at the symmetrical position with the image edge as the symmetrical axis.

[0094] The image gray value in each local feature region obtained by dividing is subjected to two-dimensional fast Fourier transform, and the gray value in the region is subjected to mean zero preprocessing before transformation, that is, the arithmetic mean value of all pixel gray values in the region is calculated, and then the gray value of each pixel is subtracted by the arithmetic mean value, and this preprocessing step can eliminate the influence of direct current component on frequency domain analysis; the two-dimensional fast Fourier transform is realized by row and column separation, that is, one-dimensional fast Fourier transform is performed on each row, and then one-dimensional fast Fourier transform is performed on each column of the transformation result, to obtain the corresponding frequency domain spectrum in complex form, the size of the frequency domain spectrum is the same as that of the local feature region, and the frequency domain spectrum includes two components of real part and imaginary part.

[0095] The energy distribution of each frequency domain spectrum is calculated, and the energy distribution is obtained by calculating the square of the modulus of each frequency component in the frequency domain spectrum; the specific calculation process is as follows: for each complex element in the frequency domain spectrum, the product of the real part value and the real part value is calculated, and the product of the imaginary part value and the imaginary part value is added, to obtain the energy value of the frequency component; all the energy values of the frequency components are combined to form a real number matrix with the same size as the frequency domain spectrum, and this matrix is the frequency energy distribution diagram of the local feature region, and the size of the energy value reflects the proportion of the frequency component in the image.

[0096] The energy distribution entropy value of each frequency domain spectrum is calculated based on the energy distribution of the corresponding frequency domain spectrum. Before the calculation, each element value in the energy distribution matrix is divided by the sum of all element values in the matrix to obtain a normalized probability distribution matrix, ensuring that the sum of all probability values is 1. The calculation process of the energy distribution entropy value is as follows: for each probability value in the normalized matrix, when the probability value is greater than zero, the negative value of the logarithmic probability value with 2 as the base is calculated, and when the probability value is equal to zero, the value of this item is zero, and then the sum of all calculation results is obtained. The size of the energy distribution entropy value reflects the concentration degree of frequency energy, and the smaller the entropy value, the more concentrated the energy is in a small number of frequency components, and the larger the entropy value, the more dispersed the energy distribution is.

[0097] The calculated energy distribution entropy value is used as a quantitative index of the distribution concentration degree of image components in the frequency domain in the corresponding local feature region. This quantitative index can effectively represent the texture characteristics of the image content. For a uniform texture background region, the frequency energy is usually dispersed, and the energy distribution entropy value is large. For a region containing defects, since the defects usually exhibit local abnormal characteristics, the frequency energy is usually concentrated on specific frequency components, and the energy distribution entropy value is small. This quantitative index can distinguish between normal regions and potential defect regions, providing an important basis for subsequent defect judgment. The entire calculation process is based on mature digital image processing algorithms, which can ensure the accuracy and reliability of the analysis results.

[0098] S6, compare the distribution concentration degree with the preset defect judgment threshold adjusted according to the artifact index to determine whether the corresponding local feature region is a defect region, and the specific implementation includes:

[0099] After the quantification of the frequency domain distribution concentration degree is completed, the defect region judgment process is performed. First, the energy distribution entropy value corresponding to each local feature region is obtained as a quantitative index of the distribution concentration degree. The energy distribution entropy value is a numerical representation obtained by performing two-dimensional fast Fourier transform on each 32 pixel x 32 pixel local feature region and calculating the frequency energy distribution entropy. At the same time, the artifact index calculated by analyzing the texture anisotropy of the uniform background region is obtained. The artifact index is a dimensionless number obtained by calculating the average value of the standard deviation of the information entropy value in four directions of multiple 16 pixel x 16 pixel local windows, reflecting the severity of the artificial artifacts introduced by the deblurring operation in the image.

[0100] The setting of the normalization weight coefficient is based on statistical analysis of a large amount of experimental data, and the specific determination method is as follows: 200 mold surface image samples of different quality grades are collected, including 50 clear images, 50 light artifact images, 50 moderate artifact images and 50 severe artifact images, the artifact index value of each image is calculated, and the misjudgment area caused by artifact is manually marked, the mathematical relationship between the artifact index value and the area proportion of the misjudgment area is established by least square regression analysis, and the 0.5 times of the reciprocal of the slope of the regression curve is taken as the normalization weight coefficient, which is usually between 0.1 and 0.3, for example, 0.2; the role of the normalization weight coefficient is to adjust the influence degree of the artifact index on the determination threshold, so as to ensure that the threshold adjustment can effectively suppress the interference of artifacts and will not excessively reduce the detection sensitivity.

[0101] The artifact index is multiplied by the normalization weight coefficient to obtain the normalized artifact influence factor; this calculation process converts the absolute value of the artifact index into a relative influence degree, so that different levels of artifact interference can appropriately affect the final determination threshold; the artifact influence factor is a dimensionless value, which is proportional to the severity of the artifact, for example, when the artifact index is 1.5 and the normalization weight coefficient is 0.2, the artifact influence factor is calculated as 1.5 x 0.2 = 0.3.

[0102] The determination of the preset defect judgment threshold is based on the statistical characteristics of normal mold surface images, and the specific method is as follows: 500 mold surface images without defects are collected, each image is divided into 32 pixel x 32 pixel local feature regions, the energy distribution entropy value of each region after two-dimensional fast Fourier transform is calculated, the distribution of these entropy values is counted, and the value at the 5th percentile position of all entropy values after sorting by size is taken as the initial preset defect judgment threshold; this threshold represents the lower limit of the normal region energy distribution entropy value, and the region below this value is likely to contain defects, for example, the threshold is usually between 2.5 and 3.5.

[0103] The preset defect judgment threshold and the normalized artifact influence factor are added to obtain the adjusted determination threshold; the significance of this addition operation is that when there is a serious artifact in the image, appropriately increasing the determination threshold can avoid misjudging the texture caused by the artifact as a real defect; the adjusted determination threshold is a dynamic value that can automatically adapt to the image quality, for example, when the preset defect judgment threshold is 3.0 and the artifact influence factor is 0.3, the adjusted determination threshold is 3.0 + 0.3 = 3.3.

[0104] The energy distribution entropy value of each local feature region is compared with the adjusted determination threshold value; the comparison is directly compared by using the numerical value, and the smaller the energy distribution entropy value, the more concentrated the frequency energy, and the more likely to contain defect features; a one-to-one correspondence is established during the comparison process, and each local feature region is compared with only one adjusted determination threshold value.

[0105] When the energy distribution entropy value of a certain local feature region is lower than the adjusted determination threshold value, the local feature region is determined as a defect region; this determination logic is based on the following principle: a normal mold surface usually has a uniform texture distribution, and the frequency energy is relatively dispersed, and the energy distribution entropy value is higher; while the defect region has abnormal structure, and the frequency energy is often concentrated on a specific frequency component, and the energy distribution entropy value is lower; through this dynamic threshold adjustment mechanism, the detection sensitivity can be ensured while effectively reducing the misjudgment risk caused by deblurring artifacts, and the accuracy and reliability of defect detection are improved; all determination results are output in the form of a binary image, in which the defect region is marked as 255, and the non-defect region is marked as 0.

[0106] Step S6 dynamically adjusts the defect determination threshold value by introducing the artifact index; the deblurring processing will introduce directional artifacts while improving the image clarity; these artifacts have similar features in the frequency domain as real defects, and if a fixed threshold is used, false positives are easy to occur. By quantifying the severity of the artifacts and adaptively adjusting the determination threshold value, an optimal balance between detection sensitivity and specificity is achieved; compared with the existing technology using a static threshold, this method effectively solves the false positive problem caused by algorithm side effects, and significantly improves the accuracy and reliability of defect recognition.

[0107] The calculations involved in the embodiments are all de-dimensioned numerical calculations, and the preset parameters and threshold values in the calculations are set by those skilled in the art according to actual conditions.

[0108] It should be noted that the present application can be deployed in the device itself to realize embedded application, or run on PC or other terminals with user interface, so as to meet various hardware environments and use requirements.

[0109] The above-described embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the above-described embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center through wireless or wired transmission. The wired transmission includes optical fiber, twisted pair, coaxial cable, etc. The wireless transmission includes infrared, microwave, etc. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0110] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and module can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0111] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the modules is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed each other can be indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0112] The modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, which can be located in one place or distributed on a plurality of network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0113] In addition, each functional module in the various embodiments of the present application can be integrated in one processing module, or each module can exist physically independently, or two or more modules can be integrated in one module.

[0114] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0115] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0116] Finally: the above is only the preferred embodiment of the present application, and is not used to limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method of visual inspection of a mold defect, characterized by, Comprise: S1, the image acquisition device carried by the mobile platform obtains the mold surface image to be detected; S2, analyze the gradient amplitude of each region in the image to be detected, and determine the blur characteristics of different regions in the image to be detected according to the difference of the gradient amplitude; S3, evaluate the expected confidence of each region to perform deblurring operation based on the blur characteristics, and perform deblurring operation on each region based on the expected confidence to obtain a preliminary restoration image; S4, calculate the information entropy value of a plurality of local windows in different directions in the uniform background area of the preliminary restoration image, and calculate the anisotropy ratio of local entropy based on the information entropy value, and the anisotropy ratio is quantified as an artifact index; S5, a plurality of local feature regions are demarcated in the preliminary restoration image, and the distribution concentration degree of the image components in the frequency domain in each local feature region is analyzed; S6, compare the distribution concentration degree with the preset defect judgment threshold adjusted according to the artifact index, and determine whether the corresponding local feature region is a defect region; The step S2 comprises: The Sobel operator is used to calculate the horizontal direction gradient value and the vertical direction gradient value of each pixel point in the image to be detected; The gradient amplitude of the corresponding pixel point is calculated according to the horizontal direction gradient value and the vertical direction gradient value of each pixel point; The image to be detected is divided into a plurality of rectangular regions with the same size, and the average value of the gradient amplitude of all pixel points in each rectangular region is calculated; According to the difference of the average value of the gradient amplitude of a plurality of rectangular regions, the blur characteristics of different regions in the image to be detected are determined, including: the region with the average value of the gradient amplitude lower than the first preset threshold is determined as the high blur region, the region with the average value of the gradient amplitude between the first preset threshold and the second preset threshold is determined as the moderate blur region, and the region with the average value of the gradient amplitude higher than the second preset threshold is determined as the low blur region; The step S4 comprises: Select the region with the low gray variance threshold in the preliminary restoration image as the uniform background area; A plurality of local windows are arranged in the uniform background area in a sliding manner; For each local window, the gray level co-occurrence matrix in four directions of zero, forty-five, ninety and one hundred and thirty-five degrees is calculated; The information entropy value of each direction is calculated based on the gray level co-occurrence matrix of each direction; For each local window, the standard deviation of the information entropy value of the four directions is calculated; The average value of the standard deviation of all local windows is quantified as an artifact index.

2. The method of claim 1, wherein, The image acquisition device carried by the mobile platform obtains the mold surface image to be detected, comprising: Control the mobile platform to move to the detection site of the mold surface along the preset scanning path; When the mobile platform stably resides at the detection site, trigger the image acquisition device to collect the image to be detected on the mold surface with the predetermined exposure parameter.

3. The method of claim 2, wherein The process of collecting the image to be detected includes controlling the light source to irradiate the mold surface with constant illumination and angle.

4. The method of claim 1, wherein, Evaluate the expected confidence of each region to perform deblurring operation based on the blur characteristics, and perform deblurring operation on each region based on the expected confidence to obtain a preliminary restoration image, comprising: For each region with determined blur characteristics, calculate the structure tensor in a plurality of different integral scales; perform eigenvalue decomposition on each structure tensor under each integral scale to extract a principal eigenvalue of each structure tensor; arrange the principal eigenvalues under all integral scales in order of scale to form an eigenvalue trajectory; calculate a standard deviation of the eigenvalue trajectory as a quantitative indicator of trajectory stability; compare the quantitative indicator of trajectory stability with a stability threshold, and assign a high confidence rating to a region where the quantitative indicator of trajectory stability is less than or equal to the stability threshold, and assign a low confidence rating to a region where the quantitative indicator of trajectory stability is higher than the stability threshold; perform standard deblurring on the high confidence rating region and perform weakened deblurring on the low confidence rating region to obtain a preliminary restored image.

5. The method of claim 4, wherein, perform standard deblurring on the high confidence rating region using a Lucy-Richardson deconvolution algorithm, and perform weakened deblurring on the low confidence rating region using a Lucy-Richardson deconvolution algorithm with reduced iteration times.

6. The method of claim 1, wherein, divide the preliminary restored image into a plurality of local feature regions, and analyze the distribution concentration of image components in the frequency domain in each local feature region, including: divide the preliminary restored image into a plurality of non-overlapping rectangular regions as local feature regions; perform two-dimensional fast Fourier transform on the image gray values in each local feature region to obtain a corresponding frequency domain spectrum; calculate the energy distribution of each frequency domain spectrum; calculate the energy distribution entropy value of the corresponding frequency domain spectrum based on the energy distribution of each frequency domain spectrum; use the calculated energy distribution entropy value as a quantitative indicator of the distribution concentration of image components in the frequency domain in the corresponding local feature region.

7. The method of claim 1, wherein, compare the distribution concentration with a preset defect judgment threshold adjusted according to the artifact index to determine whether the corresponding local feature region is a defect region, including: obtain the energy distribution entropy value corresponding to each local feature region as a quantitative indicator of the distribution concentration; obtain the artifact index; multiply the artifact index by a normalization weight coefficient to obtain a normalized artifact influence factor; add the preset defect judgment threshold and the normalized artifact influence factor to obtain an adjusted judgment threshold; compare the energy distribution entropy value of each local feature region with the adjusted judgment threshold; when the energy distribution entropy value of the local feature region is lower than the adjusted judgment threshold, determine that the corresponding local feature region is a defect region.

Images