Casting surface defect detection method based on improved yolov5

Abstract

The invention discloses a casting surface defect detection method based on improved yolov5, and the method comprises the steps: obtaining a casting surface image, and inputting the casting surface image to the improved yolov5 to obtain a defect detection result; wherein a residual high-efficiency layer aggregation module and a regional attention module are introduced into a backbone network of the improved yolov5; the regional attention module is deployed in front of the SPPF module, channel weights are generated through global pooling, and correlation between feature map channels is captured; and the residual high-efficiency layer aggregation module cooperates with the region attention module, and reduces the self-attention calculation complexity by dividing the feature map region. According to the method, the problems of missing detection and low recognition rate caused by insufficient processing of traditional YOLOv5 on tiny targets and multi-scale features can be solved, so that high-precision online detection of the surface defects of the casting is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of casting surface defect detection, and more particularly to a casting surface defect detection method based on improved yolov5. BACKGROUND

[0002] Casting is not only an industrial product, but also a component element of large machinery, which is widely used in aerospace, industrial ships, mechanical and electronic, and transportation and other fields of the national economy. With the development of society, the demand for castings shows exponential growth.

[0003] However, the production environment of castings is relatively complex, and due to design and production machine failure, etc., various defects occur in the production process of castings, which directly affects the quality and service life of the castings.

[0004] With the development of science and technology, machine vision intelligent detection is widely used in casting surface defect detection. In the industrial production process of castings, defect detection can analyze the surface quality of castings and quickly identify the type of defects.

[0005] However, the current yolov5 algorithm has limited processing capacity for small targets and multi-scale features, resulting in missed detection and poor recognition rate when detecting some small defects.

[0006] Therefore, how to overcome the above problems is a problem that those skilled in the art need to solve. SUMMARY

[0007] Therefore, in order to at least partially solve the above technical problems, the present application provides a casting surface defect detection method based on improved yolov5, which aims to improve the multi-scale feature processing capability and thus improve the accuracy of defect detection.

[0008] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0009] A casting surface defect detection method based on improved yolov5, comprising the following steps:

[0010] Obtaining a casting surface image and inputting it into an improved yolov5 to obtain a defect detection result;

[0011] The residual efficient layer aggregation module and the region attention module are introduced into the backbone network of the improved yolov5;

[0012] The region attention module is deployed before the SPPF module, generates channel weights through global pooling, and captures the correlation between feature map channels; the residual efficient layer aggregation module cooperates with the region attention module, and reduces the complexity of self-attention calculation by dividing the feature map region.

[0013] In an optional embodiment, the region attention module comprises:

[0014] An input feature conversion unit: performing channel compression on the input feature map to generate an intermediate feature map;

[0015] A bidirectional spatial attention encoding unit: splitting the intermediate feature map into two independent one-dimensional feature encoding streams along the horizontal and vertical directions, respectively, and performing the following operations:

[0016] Global pooling with direction perception to aggregate long-range dependencies in the respective spatial dimensions;

[0017] Reserving position information to generate a pair of direction-sensitive feature encoding maps;

[0018] An attention map generation unit: concatenating and fusing the two one-dimensional feature encoding maps; generating a spatial position-aware attention weight map through convolution and nonlinear activation; splitting the weight map into a horizontal direction attention map and a vertical direction attention map;

[0019] An attention weighting unit: applying the horizontal direction attention map and the vertical direction attention map to the original input feature map, respectively, to output a position-enhanced feature map.

[0020] In an optional embodiment, the residual efficient layer aggregation module comprises, connected in sequence:

[0021] An input convolution unit, a feature splitting unit, a cross-stage convolution network unit, and an output convolution unit; wherein the cross-stage convolution network unit comprises a first path and a second path in parallel,

[0022] The first path contains n Bottleneck layers connected in series for deep feature extraction;

[0023] The second path directly transmits the input feature to the output end.

[0024] In an optional embodiment, after obtaining the casting surface image, preprocessing is performed: Mosaic data enhancement, adaptive Anchor calculation, and adaptive image scaling.

[0025] In an optional embodiment, according to the defect detection result, CIOU_LOSS is used as the loss function of the bounding box to alleviate the impact of rotation and inclination on target detection performance.

[0026] In an optional embodiment, according to the defect detection result, the Softmax function is used to calculate the class probability, and through exponential operation and normalization processing, the original class score is converted into a probability value.

[0027] In an optional embodiment, the NMS non-maximum suppression is performed on the defect detection result, including sorting all predicted frame positions, selecting the predicted frame with the highest confidence, confirming it as a correct prediction, and calculating the IOU of the predicted frame with other predicted frames, removing the predicted frames with high overlap according to the calculated value, and directly deleting the predicted frames with IOU>threshold, repeating the above process until there is no remaining.

[0028] The application discloses a casting surface defect detection method based on improved yolov5.

[0029] The residual efficient layer aggregation module effectively compresses the feature map, reduces the calculation amount, better aggregates multi-scale information, and maintains or enhances the expression ability of the model, finally effectively solves the problems of missed detection and poor recognition rate caused by the insufficient processing of micro targets and multi-scale features of YOLOv5. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0031] Figure 1 The C2f module structure diagram provided by the present application.

[0032] Figure 2 The Yolov5 improved network diagram provided by the present application.

[0033] Figure 3 The Focus structure diagram provided by the present application. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0035] In the following description, a lot of specific details are set forth in order to provide a thorough understanding of the present application, however, the present application can be practiced in other manners different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the present application, therefore, the present application is not limited to the specific embodiments disclosed below.

[0036] The embodiment of the present application discloses a casting surface defect detection method based on improved yolov5, in order to obtain the characteristics of a specific task, the traditional yolov5 structure is mainly improved to highlight the key information of different tasks, and then the representation of the specific task is enhanced.

[0037] Specifically, in order to solve the bottleneck of difficult to realize accurate prediction of multi-task, a residual efficient layer aggregation module and a regional attention module are proposed, which can reduce the calculation complexity and improve the real-time performance in the extraction of feature information, optimize the feature aggregation ability of deep network, relieve gradient vanishing, and fuse the modules to realize the fusion between the feature information of each layer, generate a feature map with multi-scale information, and improve the accuracy of defect detection.

[0038] According to the embodiment of the present application, the casting surface defect detection step comprises:

[0039] Obtain the casting surface image, input it into the improved yolov5 to obtain the defect detection result;

[0040] The residual efficient layer aggregation module and the regional attention module are introduced into the backbone network of the improved yolov5;

[0041] The regional attention module is arranged before the SPPF module, generates channel weight through global pooling, captures the correlation between feature map channels, and reduces the calculation complexity of self-attention by dividing the feature map region.

[0042] The present application can realize high-precision online detection of casting surface defects and improve the accuracy of online monitoring of casting surface quality.

[0043] In an optional embodiment, the regional attention module is used to perform global average pooling on the input feature map to generate a channel description vector; the inter-channel dependency relationship is learned through a fully connected layer; and the weight is multiplied with the original feature map channel by channel to output a weighted feature map.

[0044] Specifically, it comprises:

[0045] Input feature conversion unit: performing channel compression on the input feature map to generate an intermediate feature map;

[0046] Bidirectional spatial attention encoding unit: the intermediate feature map is split into two independent one-dimensional feature encoding streams along the horizontal direction and the vertical direction, and the following operations are performed respectively:

[0047] Global pooling with direction perception, aggregating long-range dependencies in respective spatial dimensions;

[0048] Reserving position information, generating a pair of direction-sensitive feature encoding maps;

[0049] Attention map generation unit: splicing and fusing the two one-dimensional feature encoding maps; generating spatial position-aware attention weight maps through convolution and nonlinear activation; splitting the weight map into horizontal direction attention map and vertical direction attention map;

[0050] Attention weighting unit: the horizontal direction attention map and the vertical direction attention map are respectively applied to the original input feature map, and a position-enhanced feature map is output.

[0051] In an optional embodiment, the residual efficient layer aggregation module is used to solve the problems of gradient vanishing and limited representation ability in deep neural networks, mainly including two convolutional layers (Conv) connected through residual connection; in the residual connection, the input signal is directly transmitted to the output signal, and at the same time, the mapping relationship between the input and the output is learned through convolution operation in the transmission process.

[0052] Optionally, the structure is as shown in Figure 1 , comprising sequentially connected:

[0053] The input convolution unit CBS, the feature splitting unit Split, the cross-stage convolutional network unit, and the output convolution unit CBS; wherein the cross-stage convolutional network unit includes a first path and a second path in parallel,

[0054] The first path includes n bottleneck layers Bottleneck connected in series, for deep feature extraction;

[0055] The second path directly transmits the input feature to the output end.

[0056] In an exemplary embodiment, the present application designs and builds a machine vision detection platform, which cooperates with an industrial camera through a mechanical arm to perform multi-angle circumferential shooting on the surface of a casting to collect sample information of the surface of the casting, and further classifies the collected casting surface image sample data, and labels and divides them through labelme, thereby constructing a casting surface defect image sample dataset;

[0057] The pictures in the casting surface defect dataset are batched into the improved yolov5, and the model will perform centralized preprocessing on the images in the dataset through Mosaic data enhancement, adaptive image scaling, and adaptive anchor box calculation operations;

[0058] The application optimizes the backbone network of the traditional YOLO network, designs a region attention module between the Backbone backbone network and the neck network, and combines the optimized backbone network Backbone, the neck network Neck and the detection head Head to obtain an improved YOLO network model.

[0059] In this embodiment, the improved YOLO network structure is as follows Figure 2 ,

[0060] Backbone network (Backbone): after the input is subjected to Focus slicing and CBS convolution, efficient residual feature aggregation is performed through a multi-level C2f module (instead of the original C3), a CA coordinate attention module is introduced at the end, position perception information is captured through bidirectional spatial coding, and then multi-scale context is fused through SPPF;

[0061] Neck network (Neck): a bidirectional pyramid structure is adopted, deep features are upsampled and concatenated with shallow features, and cross-scale feature reuse is strengthened by a C2f module to realize multi-level fusion from top to bottom and from bottom to top;

[0062] Detection head (Head): the fused multi-scale features are input into three detection branches, each of which aggregates semantic information through a C2f module, and finally outputs positioning and classification results through Detect.

[0063] The application extracts feature information by improving the Backbone backbone network in yolov5. The extracted information is converted into multi-layer feature maps. A region attention module is designed to reduce computational complexity and improve real-time performance. A residual efficient layer aggregation module is proposed, which includes two convolution layers (Conv). The input signal is directly transmitted to the output signal through residual connection in the middle. In the residual connection, the input signal is directly transmitted to the output signal, and in the transmission process, the mapping relationship between the input and the output is learned through convolution operation, solving the problems of gradient vanishing and limited representation ability in deep neural networks, and improving the accuracy of defect detection.

[0064] To further optimize the above technical solutions, the application uses CIOU as a regression loss function of the bounding box in the defect detection task to improve the quality of the bounding box. At the same time, a Softmax function is used to calculate the probability of each class in the image. Finally, the bounding box coordinates are mapped to the original image, and through the non-maximum suppression technology, the bounding box with high confidence is selected according to the overlap between the bounding boxes, and the redundant target box is eliminated, and the final target detection result is output. Based on the final defect detection model, the surface defects of the casting are detected to determine whether the quality of the casting surface meets the production standard.

[0065] In a specific embodiment, the surface defect detection step comprises:

[0066] S1, pre-processing the acquired casting surface image:

[0067] Mosaic data enhancement is designed at the input end, four pictures are combined into one picture according to a certain proportion, so that the model can recognize the target in a small range, and the data diversity is increased;

[0068] An adaptive anchor is designed to dynamically adjust the anchor size, reduce the model search space, speed up the convergence, and improve the training speed;

[0069] Adaptive image scaling dynamically calculates the black border value to scale the original image to the target size, improving the inference speed;

[0070] S2, feature extraction:

[0071] The 608x608x3 picture is input into the Focus structure to crop the picture, and after Slice and Concat operations, it becomes a 304x304x12 feature map, and then after a convolution operation with 32 convolution kernels, it becomes a 304x304x32 feature map, as shown in Figure 3 The CA region attention module is designed before the SPPF module, which divides the channel attention into two one-dimensional feature encoding streams, aggregates long-range dependencies in two spatial directions, and preserves position information to generate feature maps and encode them into a pair of attention maps., respectively used for direction perception and position perception, to strengthen the model's attention to the detection target and more easily locate and recognize the detection target.

[0072] S3, feature fusion:

[0073] The input of the entire network is 608x608, which is then processed by the C2f module to generate a 76x76 feature map, and then after two downsampling operations, a 19x19 feature map is generated.

[0074] Then it is passed into the Neck, which performs fusion operations in turn. First, the smaller feature map layer is upsampled to adjust it to the same size, and then the two feature maps of the same size are stacked.

[0075] S4, detection result output:

[0076] CIOU_LOSS is used as the loss function of the bounding box to alleviate the impact of rotation and tilt on target detection performance;

[0077] The Softmax function is used to calculate the probability of each class in the image, which converts the original class score into a probability value through exponential operation and normalization processing;

[0078] NMS non-maximum suppression, sort all prediction box positions, select the highest confidence prediction box, confirm it as the correct prediction, and calculate the IOU of other prediction boxes, remove the high overlap according to the calculated value IOU>threshold threshold, repeat the above process until there is no left.

[0079] The improved yolov5 casting surface defect detection method provided by the application can improve the gradient flow and feature reuse efficiency through C2f, enhance the spatial position sensing ability at the end of the backbone through the CA module, and cooperatively optimize the target positioning accuracy (especially for small targets and occluded targets). In the application process, high-precision detection of casting surface defects can be realized, and the surface quality of the casting product can be monitored.

[0080] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the device disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the related parts can be referred to the method part.

[0081] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A casting surface defect detection method based on improved yolov5, characterized in that: Obtain the casting surface image and input it into the improved yolov5 to obtain the defect detection results; The improved yolov5 backbone network introduces a residual efficient layer aggregation module and a regional attention module; The regional attention module is deployed before the SPPF module, generates channel weights through global pooling, and captures the correlation between feature map channels; the residual efficient layer aggregation module cooperates with the regional attention module to reduce the complexity of self-attention calculation by dividing the feature map area.

2. The method for detecting casting surface defects according to claim 1, wherein: The regional attention module includes: Input feature conversion unit: performs channel compression on the input feature map to generate an intermediate feature map; Bidirectional spatial attention encoding unit: Split the intermediate feature map into two independent one-dimensional feature encoding streams in the horizontal and vertical directions, and perform the following operations respectively: Aggregate long-range dependencies in their respective spatial dimensions through direction-aware global pooling; Retain position information and generate a pair of direction-sensitive feature encoding maps; Attention map generation unit: concatenates and fuses the two one-dimensional feature encoding maps; generates a spatial position-aware attention weight map through convolution and nonlinear activation; and splits the weight map into a horizontal attention map and a vertical attention map; Attention weighting unit: The horizontal attention map and the vertical attention map are respectively applied to the original input feature map, and the feature map with enhanced output position is output.

3. The method for detecting casting surface defects according to claim 1, wherein: The residual efficient layer aggregation module includes the following connected in sequence: An input convolution unit, a feature splitting unit, a cross-stage convolutional network unit, and an output convolution unit; wherein the cross-stage convolutional network unit includes a first path and a second path in parallel, The first path contains n series-connected bottleneck layers for deep feature extraction; The second path passes the input features directly to the output.

4. The method for detecting casting surface defects according to claim 1, wherein: After obtaining the casting surface image, preprocessing is performed: Mosaic data enhancement, adaptive anchor calculation, and adaptive image scaling.

5. The method for detecting casting surface defects according to claim 1, wherein: According to the defect detection results, CIOU_LOSS is used as the loss function of the bounding box to alleviate the impact of rotation and tilt on target detection performance.

6. The method for detecting casting surface defects according to claim 1, wherein: The Softmax function is used to calculate the category probability based on the defect detection results, and the original category score is converted into a probability value through exponential operation and normalization.

7. The method for detecting casting surface defects according to claim 1, wherein: Perform NMS non-maximum suppression on the defect detection results.

Images