A method for detecting surface defects in metal castings based on machine vision and deep learning

By combining multi-scale Gaussian difference and convolutional networks with level set boundary evolution, the problem of identifying small and large defects in the surface defect detection of metal castings is solved, reducing false detections and improving the accuracy and adaptability of detection.

CN123118573APending Publication Date: 2026-11-17WUXI HAISHENG TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611387849.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-08
Publication Date
2026-11-17

Smart Images

  • Figure CN123118573A_ABST
    Figure CN123118573A_ABST
Patent Text Reader

Abstract

This invention discloses a method for detecting surface defects in metal castings based on machine vision and deep learning, belonging to the field of machine vision inspection technology. The method includes the following steps: S1, acquiring and preprocessing images; S2, performing multi-scale Gaussian difference to extract gray-level abrupt change regions at different scales and constructing a training sample set; S3, using a multi-scale convolutional network as the backbone and performing joint training to obtain a defect detection model; S4, performing multi-scale feature encoding and defect category identification on candidate defect regions, driving the initial contour iterative evolution based on image gradients, and correcting the defect boundary through curvature constraints; S5, extracting gray-level profiles inside and outside the boundary, verifying the recursive cumulative likelihood ratio, and filtering out false detection regions to obtain the defect detection result; S6, obtaining verification annotations and incrementally updating the defect detection model. This invention can achieve automatic identification of surface defects in metal castings, improving detection accuracy and stability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of machine vision inspection technology, and in particular to a method for detecting surface defects in metal castings based on machine vision and deep learning. Background Technology

[0002] During the forming, demolding, cleaning, and subsequent processing of metal castings, defects such as cracks, porosity, inclusions, shrinkage cavities, and sand holes are prone to appear on the surface. Some defects are small in size and have indistinct boundaries, easily blending with oxidation textures, machining marks, and localized reflections. Traditional manual visual inspection is greatly affected by personnel experience and working conditions, making it difficult to meet the needs of large-scale inspection under continuous production conditions.

[0003] Existing machine vision inspection methods typically utilize grayscale thresholding, edge detection, region growing, and morphological processing to extract suspected defect areas. However, due to uneven lighting, localized highlights, shadows, and noise interference on the surface of castings, the grayscale representation of similar defects can easily change. Fixed thresholds are insufficient to cover both small defects and low-contrast defects, and areas with abrupt changes in surface texture are also prone to false detections.

[0004] Some existing technologies use convolutional neural networks for defect classification and region segmentation, but different defects vary in size, shape, and grayscale variation, making it difficult for single-scale features to capture both fine edges and the overall shape of larger defects. Furthermore, training imbalance may occur when classification and segmentation tasks are trained together, affecting the consistency between defect category recognition and defect region segmentation.

[0005] Existing deep learning methods are susceptible to the segmentation boundaries output by feature downsampling and image noise, resulting in local gaps, jagged edges, and positional shifts. For oxide spots, reflective areas, and regular textures, relying solely on model confidence may still retain false detection regions. Furthermore, when casting materials, production processes, or acquisition conditions change, models trained on fixed datasets struggle to adapt to new defect morphologies in a timely manner. Retraining the entire model is costly, while updating with only new samples can easily lead to the forgetting of existing defect features.

[0006] Therefore, how to provide a method for detecting surface defects in metal castings based on machine vision and deep learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0007] One objective of this invention is to propose a method for detecting surface defects in metal castings based on machine vision and deep learning. This invention fully utilizes multi-scale Gaussian difference, multi-scale convolutional networks, level set boundary evolution, and sequential probability discrimination techniques. It details the implementation process of defect candidate region extraction, defect category identification, defect region segmentation, boundary correction, false detection screening, and incremental model update. It has the advantages of accurate defect identification, fine boundary localization, low false detection rate, and strong field adaptability.

[0008] The surface defect detection method for metal castings based on machine vision and deep learning according to embodiments of the present invention includes the following steps: S1. Acquire surface images of metal castings and preprocess them to generate standard image sequences; S2. Perform multi-scale Gaussian difference on the standard image sequence to extract gray-scale abrupt change regions at different scales, merge defect candidate regions based on region overlap, and construct a training sample set by combining defect annotations. S3. Based on the training sample set, a multi-scale convolutional network is used as the backbone to perform hierarchical feature extraction and cross-scale feature fusion on the training samples. The model is then jointly trained by combining classification loss and segmentation loss to obtain the defect detection model. S4. Based on the defect detection model, multi-scale feature encoding and defect category identification are performed on the defect candidate region. The segmentation boundary is used as the initial contour. The level set method is used to drive the iterative evolution of the initial contour according to the image gradient, and the defect boundary is corrected by curvature constraint. S5. Extract the grayscale profile inside and outside the boundary along the normal of the corrected defect boundary, calculate the multi-scale Gaussian difference response difference, use the sequential probability ratio test to deduce the cumulative likelihood ratio according to the boundary sampling order, and screen out false detection areas according to the upper and lower decision thresholds to obtain the defect detection results. S6. Obtain the verification annotations of the defect detection results, and use online difficult case mining to screen differential samples, and incrementally update the defect detection model by combining historical sample playback.

[0009] Optionally, the preprocessing specifically includes: performing distortion correction and detection region cropping on the surface image, adjusting the image illumination distribution using multi-scale Retinex decomposition, suppressing noise interference using bilateral filtering, unifying the size and mapping the grayscale of the processed image, and generating a standard image sequence according to the acquisition order.

[0010] Optionally, S2 specifically includes: S21. Convolve the standard image according to the increasing Gaussian scale, calculate the pixel difference between adjacent scale smoothed images, and form a multi-scale Gaussian difference result. S22. Calculate the mean and standard deviation of the absolute response of the Gaussian difference results at each scale. Set the sum of the mean and standard deviation of the absolute response as the gray-level change threshold. Retain the pixels that reach the gray-level change threshold and aggregate them into gray-level change regions according to the eight-neighbor connectivity relationship. S23. Calculate the proportion of overlapping areas of gray-scale abrupt change regions at different scales, merge the gray-scale abrupt change regions whose proportions reach the region merging threshold, and obtain the defect candidate regions. S24. Calculate the intersection-union ratio of the defect candidate region and the defect annotation region: Defect candidate regions that reach the first labeling threshold are cropped into positive samples, and classification and segmentation labels are set according to the defect labels; Defect candidate regions with an intersection-union ratio (IU) lower than the second annotation threshold are cropped as negative samples. Defect candidate regions with IU between the first and second annotation thresholds are removed to construct a training sample set, where the first annotation threshold is higher than the second annotation threshold.

[0011] Optionally, S3 specifically includes: S31. The training samples are downsampled step by step using a continuous convolution stage. Convolution kernels with different dilation rates are configured in each convolution stage to extract hierarchical features that progress from local gray-scale edges to the overall shape of the defect. S32. Restore the deep-level features to the spatial scale of the adjacent shallow-level features step by step. Calculate the convolution sampling offset from the adjacent-level features. Use deformable convolution to resample according to the convolution sampling offset and complete the cross-scale feature fusion. S33. Based on cross-scale fusion features, defect category identification and defect region segmentation are performed. Focus loss is used to constrain defect category identification, and Dice loss is used to constrain defect region segmentation. S34. Calculate the gradient norms of the focus loss and Dice loss with respect to the shared convolutional parameters. Adjust the training weights of the two losses according to the difference in gradient norms, and iteratively update the network parameters through backpropagation to obtain the defect detection model.

[0012] Optionally, S32 specifically includes: S321. Perform bilinear interpolation on the deep-level features at twice the scale to make the spatial size of the deep-level features consistent with that of the adjacent shallow-level features, and adjust the number of channels to 128 by using 1×1 convolution. S322. Calculate the feature difference between deep-level features and adjacent shallow-level features at corresponding spatial locations. Use 3×3 convolution to map the feature difference into 18 offset channels. Each pair of offset channels represents the horizontal offset and vertical offset of a sampling location, respectively. S323. Limit the horizontal and vertical offsets to a range of -2 to 2 pixels, perform 3×3 deformable convolution on the 9 sampled positions after offset, and use bilinear interpolation to calculate the feature values ​​at non-integer positions. S324. Arrange the resampled deep-level features and adjacent shallow-level features along the channel direction, and process the corresponding spatial neighborhoods with two layers of 3×3 convolution in sequence, and form cross-scale fusion features step by step in the order from deep to shallow.

[0013] Optionally, S4 specifically includes: S41. Preprocess the surface image to be detected and perform multi-scale Gaussian difference to extract gray-level abrupt change regions at different scales, and form candidate regions for defects to be detected based on the overlap ratio of the gray-level abrupt change regions. S42. Using a defect detection model, multi-scale feature encoding is performed on the candidate region of the defect to be detected, the defect category is identified and the defect region is segmented, the segmentation boundary is tracked along the outermost pixel of the segmented region, and the segmentation boundary is set as the initial contour. S43. Perform Euclidean distance transformation on the interior and exterior of the initial contour, establish a signed distance function based on the difference between the interior and exterior distances, delineate narrow evolution regions on both sides of the zero horizontal line, and use horizontal gradient operators and vertical gradient operators to process the candidate regions of defects to be detected, forming edge stopping terms. S44. Within the narrow evolution region, the sign distance function is updated using windward differential iteration. The zero horizontal line is pulled by the edge stopping term to move towards the gray-scale abrupt change position, and the local protrusions and depressions are restricted by the average curvature term until the displacement of the zero horizontal line in two adjacent iterations is lower than the convergence threshold, thus obtaining the corrected defect boundary.

[0014] Optionally, S44 specifically includes: S441. Select a narrow band evolution region with a width of 5 pixels inside and outside the zero horizontal line, and obtain the sign distance value of the left, right, top and bottom adjacent pixels one by one to form the forward change amount and backward change amount in the horizontal and vertical directions respectively. S442. When the edge stop item drives the zero horizontal line to move outward, select the amount of change pointing outward from the contour. When the edge stop item drives the zero horizontal line to move inward, select the amount of change pointing inward to the contour, and combine the amount of change in the two directions into the contour movement amount of the current pixel; S443. Compare the distance values ​​of adjacent symbols in the 3×3 neighborhood of each pixel, correct the contour movement by shrinking inward at the protruding part of the contour and compensating outward at the concave part of the contour, update the symbol distance value with a time step of 0.25 to 0.5, and limit the contour displacement corresponding to a single update to within 1 pixel. S444. After every 5 rounds of updates, re-extract the positions where the symbol distance value changes positive or negative, and redefine the narrowband evolution region with the extracted position as the center. Stop the iteration when the maximum displacement of the zero horizontal line in 3 consecutive rounds of updates is less than 0.5 pixels, and extract the zero horizontal line as the corrected defect boundary.

[0015] Optionally, S5 specifically includes: S51. Sampling points are set sequentially along the corrected defect boundary. Gray-scale sequences inside and outside the boundary are extracted along the boundary normal of each sampling point to form gray-scale profiles inside and outside the boundary corresponding to the position. S52. Using an incrementally scaled Gaussian kernel, the grayscale profile inside the boundary and the grayscale profile outside the boundary are processed separately. The processing results of adjacent scales are subtracted to extract the response peaks on both sides of the boundary at each scale, and the boundary response sequence is formed by the difference of the response peaks at the same scale. S53. Based on the boundary response sequence, it is sequentially mapped to the defect boundary response distribution and the non-defect boundary response distribution. The log ratio of the corresponding probabilities of the two response distributions is used as the discrimination increment, and the cumulative likelihood ratio is recursively calculated according to the boundary sampling order. S54. When the cumulative likelihood ratio reaches the upper decision threshold, the corresponding defect area is retained. When the cumulative likelihood ratio reaches the lower decision threshold, the corresponding false detection area is screened out. When the cumulative likelihood ratio is between the upper and lower decision thresholds, the next sampling point is processed to obtain the defect detection result.

[0016] Optionally, S52 specifically includes: S521. Using one-dimensional Gaussian kernels with lengths of 5 pixels, 9 pixels, and 13 pixels respectively, convolve the gray-level profiles inside and outside the boundary to form smooth profiles at three scales. S522. Subtract each point from the smooth profiles of adjacent scales to form two sets of Gaussian difference profiles, one inside the boundary and one outside the boundary. S523. Using the boundary sampling point as the center, search for the location of the maximum Gaussian difference absolute response within a range of 3 pixels inside and outside the boundary, and extract the peak response inside and outside the boundary at the same scale. S524. Subtract the peak response inside the boundary from the peak response outside the boundary at the same scale, select the response difference with the larger absolute value among the two sets of response differences as the boundary response value of the current boundary sampling point, and form a boundary response sequence according to the boundary sampling order.

[0017] Optionally, S6 specifically includes: S61. Establish a correspondence between the defect detection results and the verification annotations, extract the detection areas where the defect categories are inconsistent, and extract the detection areas where the intersection ratio of the segmented area and the verification annotation area is lower than the difference threshold to form difference samples. S62. The difference samples are reprocessed using a defect detection model. The difference samples are sorted according to the defect category identification loss and the defect region segmentation loss. The difference samples with the highest loss values ​​are retained through online hard case mining to form hard case samples. S63. According to the defect category of the difficult sample, extract the corresponding historical samples and background samples from the training sample set, and form an incremental training batch with the same number of difficult samples and historical samples. S64. Keep the parameters of the hierarchical feature extraction stage in the defect detection model unchanged, update the corresponding parameters of cross-scale feature fusion, defect category identification and defect region segmentation by incremental training batch, and add the difficult sample to the training sample set after the update.

[0018] The beneficial effects of this invention are: First, by performing multi-scale processing on the surface image of the casting, this invention can identify both small and large defects, reduce interference from changes in lighting, surface texture, and local reflections, and improve the accuracy of defect detection.

[0019] Secondly, the present invention can simultaneously complete defect category judgment and defect area location, and further correct the initial detection boundary to make the defect contour more complete and smooth, and reduce boundary gaps, jagged edges and positional offsets.

[0020] Finally, this invention can eliminate false detections caused by oxidation spots, reflective areas and regular textures by combining the grayscale changes on both sides of the boundary, and continuously update the detection model using the verification results, so that the detection method can better adapt to changes in materials, processes and collection conditions, and improve detection stability and field applicability. Attached Figure Description

[0021] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is an overall flowchart of the surface defect detection method for metal castings based on machine vision and deep learning proposed in this invention. Figure 2 This is a flowchart of the defect detection model training process for the surface defect detection method for metal castings based on machine vision and deep learning proposed in this invention. Figure 3 This is a flowchart of the defect boundary response discrimination and false detection area screening process of the metal casting surface defect detection method based on machine vision and deep learning proposed in this invention. Detailed Implementation

[0022] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0023] refer to Figures 1-3 A method for detecting surface defects in metal castings based on machine vision and deep learning includes the following steps: S1. Acquire surface images of metal castings and preprocess them to generate standard image sequences; S2. Perform multi-scale Gaussian difference on the standard image sequence to extract gray-scale abrupt change regions at different scales, merge defect candidate regions based on region overlap, and construct a training sample set by combining defect annotations. S3. Based on the training sample set, a multi-scale convolutional network is used as the backbone to perform hierarchical feature extraction and cross-scale feature fusion on the training samples. The model is then jointly trained by combining classification loss and segmentation loss to obtain the defect detection model. S4. Based on the defect detection model, multi-scale feature encoding and defect category identification are performed on the defect candidate region. The segmentation boundary is used as the initial contour. The level set method is used to drive the iterative evolution of the initial contour according to the image gradient, and the defect boundary is corrected by curvature constraint. S5. Extract the grayscale profile inside and outside the boundary along the normal of the corrected defect boundary, calculate the multi-scale Gaussian difference response difference, use the sequential probability ratio test to deduce the cumulative likelihood ratio according to the boundary sampling order, and screen out false detection areas according to the upper and lower decision thresholds to obtain the defect detection results. S6. Obtain the verification annotations of the defect detection results, and use online difficult case mining to screen differential samples, and incrementally update the defect detection model by combining historical sample playback.

[0024] In this embodiment, the preprocessing specifically includes: performing distortion correction and detection area cropping on the surface image, adjusting the image illumination distribution using multi-scale Retinex decomposition, suppressing noise interference using bilateral filtering, unifying the size and mapping the grayscale of the processed image, and generating a standard image sequence according to the acquisition order. Acquire a surface image of a metal casting with a resolution of 2448×2048 pixels, correct radial and tangential distortions according to camera calibration parameters, and crop a detection area of ​​1024×1024 pixels from the corrected surface image; The detection region was decomposed using multi-scale Retinex decomposition with Gaussian scales of 15, 80 and 250, and the image brightness distribution was restored by averaging the reflection components at each scale. The image after illumination adjustment is processed using a bilateral filter with a neighborhood size of 7×7 pixels to preserve the gray-level abrupt changes at the defect edges; The processed image is scaled to 512×512 pixels, and the pixel grayscale values ​​are mapped to 0 to 255. A standard image sequence is generated according to the acquisition order of the surface images.

[0025] In this embodiment, S2 specifically includes: S21. Convolve the standard image according to the increasing Gaussian scale, calculate the pixel difference between adjacent scale smoothed images, and form a multi-scale Gaussian difference result. S22. Calculate the mean and standard deviation of the absolute response of the Gaussian difference results at each scale. Set the sum of the mean and standard deviation of the absolute response as the gray-level change threshold. Retain the pixels that reach the gray-level change threshold and aggregate them into gray-level change regions according to the eight-neighbor connectivity relationship. S23. Calculate the proportion of overlapping areas of gray-scale abrupt change regions at different scales, merge the gray-scale abrupt change regions whose proportions reach the region merging threshold, and obtain the defect candidate regions. S24. Calculate the intersection-union ratio of the defect candidate region and the defect annotation region: Defect candidate regions that reach the first labeling threshold are cropped into positive samples, and classification and segmentation labels are set according to the defect labels; Defect candidate regions with an intersection-union ratio (IU) lower than the second annotation threshold are cropped as negative samples. Defect candidate regions with IU between the first and second annotation thresholds are removed to construct a training sample set, where the first annotation threshold is higher than the second annotation threshold.

[0026] In this embodiment, S3 specifically includes: S31. The training samples are downsampled step by step using a continuous convolution stage. Convolution kernels with different dilation rates are configured in each convolution stage to extract hierarchical features that progress from local gray-scale edges to the overall shape of the defect. S32. Restore the deep-level features to the spatial scale of the adjacent shallow-level features step by step. Calculate the convolution sampling offset from the adjacent-level features. Use deformable convolution to resample according to the convolution sampling offset and complete the cross-scale feature fusion. S33. Based on cross-scale fusion features, defect category identification and defect region segmentation are performed. Focus loss is used to constrain defect category identification, and Dice loss is used to constrain defect region segmentation. S34. Calculate the gradient norms of the focus loss and Dice loss with respect to the shared convolutional parameters. Adjust the training weights of the two losses based on the difference in gradient norms, and iteratively update the network parameters through backpropagation to obtain the defect detection model, specifically including: Backpropagation is performed on the focus loss and Dice loss respectively, and the gradients of the two losses applied to the shared convolution parameters are extracted to form the gradients for the classification task and the segmentation task. The current values ​​of the focus loss and Dice loss are associated with their corresponding initial values, and the target magnitudes of the classification task gradient and the segmentation task gradient are adjusted according to the decrease ratio of the two losses. The training weights of the two losses are corrected by adjusting the deviations of the classification task gradient and the segmentation task gradient relative to the target magnitude, so that the update magnitudes of the two types of gradients to the shared convolution parameters gradually become closer. By combining the corrected training weights with backpropagation focus loss and Dice loss, the shared convolution parameters, defect category identification parameters, and defect region segmentation parameters are iteratively updated to obtain the defect detection model.

[0027] In this embodiment, S32 specifically includes: S321. Perform bilinear interpolation on the deep-level features at twice the scale to make the spatial size of the deep-level features consistent with that of the adjacent shallow-level features, and adjust the number of channels to 128 by using 1×1 convolution. S322. Calculate the feature difference between deep-level features and adjacent shallow-level features at corresponding spatial locations. Use 3×3 convolution to map the feature difference into 18 offset channels. Each pair of offset channels represents the horizontal offset and vertical offset of a sampling location, respectively. S323. Limit the horizontal and vertical offsets to a range of -2 to 2 pixels, perform 3×3 deformable convolution on the 9 sampled positions after offset, and use bilinear interpolation to calculate the feature values ​​at non-integer positions. S324. Arrange the resampled deep-level features and adjacent shallow-level features along the channel direction, and process the corresponding spatial neighborhoods with two layers of 3×3 convolution in sequence, and form cross-scale fusion features step by step in the order from deep to shallow.

[0028] In this embodiment, S4 specifically includes: S41. Preprocess the surface image to be detected and perform multi-scale Gaussian difference to extract gray-level abrupt change regions at different scales, and form candidate regions for defects to be detected based on the overlap ratio of the gray-level abrupt change regions. S42. Using a defect detection model, multi-scale feature encoding is performed on the candidate region of the defect to be detected, the defect category is identified and the defect region is segmented, the segmentation boundary is tracked along the outermost pixel of the segmented region, and the segmentation boundary is set as the initial contour. S43. Perform Euclidean distance transformation on the interior and exterior of the initial contour, establish a signed distance function based on the difference between the interior and exterior distances, delineate narrow evolution regions on both sides of the zero horizontal line, and use horizontal gradient operators and vertical gradient operators to process the candidate regions of defects to be detected, forming edge stopping terms. S44. Within the narrow evolution region, the sign distance function is updated using windward differential iteration. The zero horizontal line is pulled by the edge stopping term to move towards the gray-scale abrupt change position, and the local protrusions and depressions are restricted by the average curvature term until the displacement of the zero horizontal line in two adjacent iterations is lower than the convergence threshold, thus obtaining the corrected defect boundary.

[0029] In this embodiment, S44 specifically includes: S441. Select a narrow band evolution region with a width of 5 pixels inside and outside the zero horizontal line, and obtain the sign distance value of the left, right, top and bottom adjacent pixels one by one to form the forward change amount and backward change amount in the horizontal and vertical directions respectively. S442. When the edge stop item drives the zero horizontal line to move outward, select the amount of change pointing outward from the contour. When the edge stop item drives the zero horizontal line to move inward, select the amount of change pointing inward to the contour, and combine the amount of change in the two directions into the contour movement amount of the current pixel; S443. Compare the distance values ​​of adjacent symbols in the 3×3 neighborhood of each pixel, correct the contour movement by shrinking inward at the protruding part of the contour and compensating outward at the concave part of the contour, update the symbol distance value with a time step of 0.25 to 0.5, and limit the contour displacement corresponding to a single update to within 1 pixel. S444. After every 5 rounds of updates, re-extract the positions where the symbol distance value changes positive or negative, and redefine the narrowband evolution region with the extracted position as the center. Stop the iteration when the maximum displacement of the zero horizontal line in 3 consecutive rounds of updates is less than 0.5 pixels, and extract the zero horizontal line as the corrected defect boundary.

[0030] In this embodiment, S5 specifically includes: S51. Sampling points are set sequentially along the corrected defect boundary. Gray-scale sequences inside and outside the boundary are extracted along the boundary normal of each sampling point to form gray-scale profiles inside and outside the boundary corresponding to the position. S52. Using an incrementally scaled Gaussian kernel, the grayscale profile inside the boundary and the grayscale profile outside the boundary are processed separately. The processing results of adjacent scales are subtracted to extract the response peaks on both sides of the boundary at each scale, and the boundary response sequence is formed by the difference of the response peaks at the same scale. S53. Based on the boundary response sequence, it is sequentially mapped to the defect boundary response distribution and the non-defect boundary response distribution. The log ratio of the corresponding probabilities of the two response distributions is used as the discrimination increment, and the cumulative likelihood ratio is recursively calculated according to the boundary sampling order. S54. When the cumulative likelihood ratio reaches the upper decision threshold, the corresponding defect area is retained. When the cumulative likelihood ratio reaches the lower decision threshold, the corresponding false detection area is screened out. When the cumulative likelihood ratio is between the upper and lower decision thresholds, the next sampling point is processed to obtain the defect detection result.

[0031] In this embodiment, S52 specifically includes: S521. Using one-dimensional Gaussian kernels with lengths of 5 pixels, 9 pixels, and 13 pixels respectively, convolve the gray-level profiles inside and outside the boundary to form smooth profiles at three scales. S522. Subtract each point from the smooth profiles of adjacent scales to form two sets of Gaussian difference profiles, one inside the boundary and one outside the boundary. S523. Using the boundary sampling point as the center, search for the location of the maximum Gaussian difference absolute response within a range of 3 pixels inside and outside the boundary, and extract the peak response inside and outside the boundary at the same scale. S524. Subtract the peak response inside the boundary from the peak response outside the boundary at the same scale, select the response difference with the larger absolute value among the two sets of response differences as the boundary response value of the current boundary sampling point, and form a boundary response sequence according to the boundary sampling order.

[0032] In this embodiment, S6 specifically includes: S61. Establish a correspondence between the defect detection results and the verification annotations, extract the detection areas where the defect categories are inconsistent, and extract the detection areas where the intersection ratio of the segmented area and the verification annotation area is lower than the difference threshold to form difference samples. S62. The difference samples are reprocessed using a defect detection model. The difference samples are sorted according to the defect category identification loss and the defect region segmentation loss. The difference samples with the highest loss values ​​are retained through online hard case mining to form hard case samples. S63. According to the defect category of the difficult sample, extract the corresponding historical samples and background samples from the training sample set, and form an incremental training batch with the same number of difficult samples and historical samples. S64. Keep the parameters of the hierarchical feature extraction stage in the defect detection model unchanged, update the corresponding parameters of cross-scale feature fusion, defect category identification and defect region segmentation by incremental training batch, and add the difficult sample to the training sample set after the update.

[0033] Example 1: To verify the feasibility of this invention in practice, it was applied to the surface quality inspection of a certain type of pump body metal casting. After sand removal, shot peening, and cleaning, these castings are prone to developing cracks, porosity, sand holes, inclusions, and shrinkage cavities on their surface, accompanied by oxide spots, shot peening texture, residual sand particles, and localized reflections. In actual inspection, fine cracks and scratches are similar in appearance, light-colored pores are difficult to identify in reflective areas, and oxide spots are easily misjudged as defects. The stability and consistency of manual inspection are poor.

[0034] This embodiment collected 4800 images of the casting surface with a resolution of 2448×2048 pixels. 3600 images were used for training, 600 for observation during training, and 600 for final testing. A total of 686 actual defects were marked in the test images, including 126 cracks, 184 porosities, 207 sand holes, 96 inclusions, and 73 shrinkage cavities. The defect areas ranged from 18 to 4600 pixels.

[0035] After the image enters the inspection program, imaging distortion is first corrected and the surface of the main casting is cropped. Then, the brightness of dark and bright areas is adjusted, particle noise is filtered out, and the image is uniformly processed to 512×512 pixels. The processed image can retain the grayscale variations near cracks, pores, and sand holes, while reducing interference caused by shadows, reflections, and shot peening highlights.

[0036] During training, suspected defect areas are extracted from images, and 12,460 real defect samples and 16,800 background samples are generated based on manual annotation. The model simultaneously learns the local edges and overall morphology of defects, and can identify slender cracks, near-circular pores, and irregular inclusions, and provide the defect category and the location of the defect.

[0037] To address the issues of gaps and jagged edges in the initial detection boundaries, the contours were further refined based on the grayscale changes around the defects. After refinement, the average deviation between the defect boundaries and the manually marked boundaries decreased from 2.6 pixels to 1.4 pixels, and the region intersection-union ratio increased from 83.1% to 89.4%. For suspected defects formed by oxide spots, regular textures, and reflective areas, further discrimination was performed by analyzing the grayscale changes on both sides of the boundaries, reducing the number of false detection areas from 69 to 21.

[0038] Another 200 images with reduced brightness and deepened surface texture were selected for verification, and the model was updated using samples with large recognition errors and boundary deviations. Before the update, the defect recall rate was 91.9% and the false positive rate was 4.5%; after the update, the defect recall rate reached 95.0%, the false positive rate decreased to 3.0%, and the accuracy of the original test images remained at 95.9%.

[0039] Table 1 Comparison of Surface Defect Detection Results for Metal Castings

[0040] As can be seen from the data in Table 1, the present invention can effectively improve the accuracy of identifying surface defects in metal castings and the precision of boundary positioning, reduce false detections caused by oxide spots, reflections and surface textures, and take into account both detection efficiency and stability in field applications.

[0041] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting surface defects in metal castings based on machine vision and deep learning, characterized in that, Includes the following steps: S1. Acquire surface images of metal castings and preprocess them to generate standard image sequences; S2. Perform multi-scale Gaussian difference on the standard image sequence to extract gray-scale abrupt change regions at different scales, merge defect candidate regions based on region overlap, and construct a training sample set by combining defect annotations. S3. Based on the training sample set, a multi-scale convolutional network is used as the backbone to perform hierarchical feature extraction and cross-scale feature fusion on the training samples. The model is then jointly trained by combining classification loss and segmentation loss to obtain the defect detection model. S4. Based on the defect detection model, multi-scale feature encoding and defect category identification are performed on the defect candidate region. The segmentation boundary is used as the initial contour. The level set method is used to drive the iterative evolution of the initial contour according to the image gradient, and the defect boundary is corrected by curvature constraint. S5. Extract the grayscale profile inside and outside the boundary along the normal of the corrected defect boundary, calculate the multi-scale Gaussian difference response difference, use the sequential probability ratio test to deduce the cumulative likelihood ratio according to the boundary sampling order, and screen out false detection areas according to the upper and lower decision thresholds to obtain the defect detection results. S6. Obtain the verification annotations of the defect detection results, and use online difficult case mining to screen differential samples, and incrementally update the defect detection model by combining historical sample playback.

2. The method for detecting surface defects in metal castings based on machine vision and deep learning according to claim 1, characterized in that, The preprocessing specifically includes: performing distortion correction and detection region cropping on the surface image, adjusting the image illumination distribution using multi-scale Retinex decomposition, suppressing noise interference using bilateral filtering, unifying the size and mapping the grayscale of the processed image, and generating a standard image sequence according to the acquisition order.

3. The method for detecting surface defects in metal castings based on machine vision and deep learning according to claim 1, characterized in that, S2 specifically includes: S21. Convolve the standard image according to the increasing Gaussian scale, calculate the pixel difference between adjacent scale smoothed images, and form a multi-scale Gaussian difference result. S22. Calculate the mean and standard deviation of the absolute response of the Gaussian difference results at each scale. Set the sum of the mean and standard deviation of the absolute response as the gray-level change threshold. Retain the pixels that reach the gray-level change threshold and aggregate them into gray-level change regions according to the eight-neighbor connectivity relationship. S23. Calculate the proportion of overlapping areas of gray-scale abrupt change regions at different scales, merge the gray-scale abrupt change regions whose proportions reach the region merging threshold, and obtain the defect candidate regions. S24. Calculate the intersection-union ratio of the defect candidate region and the defect annotation region: Defect candidate regions that reach the first labeling threshold are cropped into positive samples, and classification and segmentation labels are set according to the defect labels; Defect candidate regions with an intersection-union ratio (IU) lower than the second annotation threshold are cropped as negative samples. Defect candidate regions with IU between the first and second annotation thresholds are removed to construct a training sample set, where the first annotation threshold is higher than the second annotation threshold.

4. The method for detecting surface defects in metal castings based on machine vision and deep learning according to claim 1, characterized in that, S3 specifically includes: S31. The training samples are downsampled step by step using a continuous convolution stage. Convolution kernels with different dilation rates are configured in each convolution stage to extract hierarchical features that progress from local gray-scale edges to the overall shape of the defect. S32. Restore the deep-level features to the spatial scale of the adjacent shallow-level features step by step. Calculate the convolution sampling offset from the adjacent-level features. Use deformable convolution to resample according to the convolution sampling offset and complete the cross-scale feature fusion. S33. Based on cross-scale fusion features, defect category identification and defect region segmentation are performed. Focus loss is used to constrain defect category identification, and Dice loss is used to constrain defect region segmentation. S34. Calculate the gradient norms of the focus loss and Dice loss with respect to the shared convolutional parameters. Adjust the training weights of the two losses according to the difference in gradient norms, and iteratively update the network parameters through backpropagation to obtain the defect detection model.

5. The method for detecting surface defects in metal castings based on machine vision and deep learning according to claim 4, characterized in that, Specifically, S32 includes: S321. Perform bilinear interpolation on the deep-level features at twice the scale to make the spatial size of the deep-level features consistent with that of the adjacent shallow-level features, and adjust the number of channels to 128 by using 1×1 convolution. S322. Calculate the feature difference between deep-level features and adjacent shallow-level features at corresponding spatial locations. Use 3×3 convolution to map the feature difference into 18 offset channels. Each pair of offset channels represents the horizontal offset and vertical offset of a sampling location, respectively. S323. Limit the horizontal and vertical offsets to a range of -2 to 2 pixels, perform 3×3 deformable convolution on the 9 sampled positions after offset, and use bilinear interpolation to calculate the feature values ​​at non-integer positions. S324. Arrange the resampled deep-level features and adjacent shallow-level features along the channel direction, and process the corresponding spatial neighborhoods with two layers of 3×3 convolution in sequence, and form cross-scale fusion features step by step in the order from deep to shallow.

6. The method for detecting surface defects in metal castings based on machine vision and deep learning according to claim 1, characterized in that, S4 specifically includes: S41. Preprocess the surface image to be detected and perform multi-scale Gaussian difference to extract gray-level abrupt change regions at different scales, and form candidate regions for defects to be detected based on the overlap ratio of the gray-level abrupt change regions. S42. Using a defect detection model, multi-scale feature encoding is performed on the candidate region of the defect to be detected, the defect category is identified and the defect region is segmented, the segmentation boundary is tracked along the outermost pixel of the segmented region, and the segmentation boundary is set as the initial contour. S43. Perform Euclidean distance transformation on the interior and exterior of the initial contour, establish a signed distance function based on the difference between the interior and exterior distances, delineate narrow evolution regions on both sides of the zero horizontal line, and use horizontal gradient operators and vertical gradient operators to process the candidate regions of defects to be detected, forming edge stopping terms. S44. Within the narrow evolution region, the sign distance function is updated using windward differential iteration. The zero horizontal line is pulled by the edge stopping term to move towards the gray-scale abrupt change position, and the local protrusions and depressions are restricted by the average curvature term until the displacement of the zero horizontal line in two adjacent iterations is lower than the convergence threshold, thus obtaining the corrected defect boundary.

7. The method for detecting surface defects in metal castings based on machine vision and deep learning according to claim 6, characterized in that, S44 specifically includes: S441. Select a narrow band evolution region with a width of 5 pixels inside and outside the zero horizontal line, and obtain the sign distance value of the left, right, top and bottom adjacent pixels one by one to form the forward change amount and backward change amount in the horizontal and vertical directions respectively. S442. When the edge stop item drives the zero horizontal line to move outward, select the amount of change pointing outward from the contour. When the edge stop item drives the zero horizontal line to move inward, select the amount of change pointing inward to the contour, and combine the amount of change in the two directions into the contour movement amount of the current pixel; S443. Compare the distance values ​​of adjacent symbols in the 3×3 neighborhood of each pixel, correct the contour movement by shrinking inward at the protruding part of the contour and compensating outward at the concave part of the contour, update the symbol distance value with a time step of 0.25 to 0.5, and limit the contour displacement corresponding to a single update to within 1 pixel. S444. After every 5 rounds of updates, re-extract the positions where the symbol distance value changes positive or negative, and redefine the narrowband evolution region with the extracted position as the center. Stop the iteration when the maximum displacement of the zero horizontal line in 3 consecutive rounds of updates is less than 0.5 pixels, and extract the zero horizontal line as the corrected defect boundary.

8. The method for detecting surface defects in metal castings based on machine vision and deep learning according to claim 1, characterized in that, S5 specifically includes: S51. Sampling points are set sequentially along the corrected defect boundary. Gray-scale sequences inside and outside the boundary are extracted along the boundary normal of each sampling point to form gray-scale profiles inside and outside the boundary corresponding to the position. S52. Using an incrementally scaled Gaussian kernel, the grayscale profile inside the boundary and the grayscale profile outside the boundary are processed separately. The processing results of adjacent scales are subtracted to extract the response peaks on both sides of the boundary at each scale, and the boundary response sequence is formed by the difference of the response peaks at the same scale. S53. Based on the boundary response sequence, it is sequentially mapped to the defect boundary response distribution and the non-defect boundary response distribution. The log ratio of the corresponding probabilities of the two response distributions is used as the discrimination increment, and the cumulative likelihood ratio is recursively calculated according to the boundary sampling order. S54. When the cumulative likelihood ratio reaches the upper decision threshold, the corresponding defect area is retained. When the cumulative likelihood ratio reaches the lower decision threshold, the corresponding false detection area is screened out. When the cumulative likelihood ratio is between the upper and lower decision thresholds, the next sampling point is processed to obtain the defect detection result.

9. The method for detecting surface defects in metal castings based on machine vision and deep learning according to claim 8, characterized in that, Specifically, S52 includes: S521. Using one-dimensional Gaussian kernels with lengths of 5 pixels, 9 pixels, and 13 pixels respectively, convolve the gray-level profiles inside and outside the boundary to form smooth profiles at three scales. S522. Subtract each point from the smooth profiles of adjacent scales to form two sets of Gaussian difference profiles, one inside the boundary and one outside the boundary. S523. Using the boundary sampling point as the center, search for the location of the maximum Gaussian difference absolute response within a range of 3 pixels inside and outside the boundary, and extract the peak response inside and outside the boundary at the same scale. S524. Subtract the peak response inside the boundary from the peak response outside the boundary at the same scale, select the response difference with the larger absolute value among the two sets of response differences as the boundary response value of the current boundary sampling point, and form a boundary response sequence according to the boundary sampling order.

10. The method for detecting surface defects in metal castings based on machine vision and deep learning according to claim 1, characterized in that, S6 specifically includes: S61. Establish a correspondence between the defect detection results and the verification annotations, extract the detection areas where the defect categories are inconsistent, and extract the detection areas where the intersection ratio of the segmented area and the verification annotation area is lower than the difference threshold to form difference samples. S62. The difference samples are reprocessed using a defect detection model. The difference samples are sorted according to the defect category identification loss and the defect region segmentation loss. The difference samples with the highest loss values ​​are retained through online hard case mining to form hard case samples. S63. According to the defect category of the difficult sample, extract the corresponding historical samples and background samples from the training sample set, and form an incremental training batch with the same number of difficult samples and historical samples. S64. Keep the parameters of the hierarchical feature extraction stage in the defect detection model unchanged, update the corresponding parameters of cross-scale feature fusion, defect category identification and defect region segmentation by incremental training batch, and add the difficult sample to the training sample set after the update.