Steel rolling surface defect detection method of improved group convolution and feature enhancement fusion network based on multi-head attention mechanism

Abstract

The invention provides a rolled steel surface defect detection method based on an improved group convolution and feature enhancement fusion network of a multi-head attention mechanism, and the method comprises the steps: obtaining a rolled steel surface defect image data set; training a defect detection model according to the steel rolling surface defect image data set based on a grouping space multi-scale convolution module and a feature enhancement fusion network; and inputting a to-be-detected image into the defect detection model to obtain a target rolled steel surface defect detection result. According to the steel rolling surface defect detection method based on the multi-head attention mechanism improved group convolution and feature enhancement fusion network, a group space multi-scale convolution module and a feature enhancement fusion network improved YOLOv7 target detection model are introduced, and a pre-trained defect detection model is obtained; the pre-training defect detection model is trained based on the training sample to obtain the defect detection model, the image needing to be detected is input into the defect detection model, the defect detection result is obtained, and the model defect detection performance is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of rolled steel plate defect detection, and in particular to a rolled steel surface defect detection method based on an improved group convolution and feature enhancement fusion network with a multi-head attention mechanism. Background Art

[0002] The steel rolling industry is an important part of the global energy industry. The steel rolled in the industry is an important raw material and is widely used in high-precision fields such as construction, machinery, aerospace, and medical treatment. During steel rolling, the rotating rollers are used to achieve plastic deformation of the metal and change the size and shape of the ingot or continuous casting billet. The steel rolling process is affected by material properties, process parameters, equipment status, etc. In order to ensure the reliability of subsequent applications, defect detection of the produced steel is required.

[0003] The invention patent with application number: 202111397969.3 discloses a method for detecting surface defects of hot-rolled steel plates, the method comprising: collecting defect images of hot-rolled steel plates, marking the defect coordinates and defect categories in the defect images, and then performing data amplification and data enhancement, and then training the pre-established YOLOSteel model to obtain the trained YOLOSteel model, and performing defect location and defect classification on the hot-rolled steel plates with defects according to the trained YOLOSteel model. The YOLOSteel model is an improved model based on the YOLOv4 network model. The YOLOSteel model is used for surface defect detection of hot-rolled steel plates. On the one hand, it reduces the number of parameters and the amount of calculation of the model, improves the detection speed, and meets the real-time nature of hot-rolled steel plate defect detection in industrial production. On the other hand, it improves the feature extraction capability of hot-rolled steel plates.

[0004] However, although the above-mentioned existing technology uses multi-scale features for convolution, the feature fusion is based on the original neck network of the YOLOv4 network model, ignoring the underlying context information, and the expression of deep-level abstract features is insufficient. The detection performance is insufficient for difficult-to-detect defects such as fine cracks and slight flaws.

[0005] In view of this, there is an urgent need for a steel rolling surface defect detection method based on an improved group convolution and feature enhancement fusion network with a multi-head attention mechanism to at least solve the above-mentioned shortcomings. Summary of the invention

[0006] One of the purposes of the present invention is to provide a method for detecting surface defects of rolled steel based on an improved group convolution and feature enhancement fusion network with a multi-head attention mechanism, introduce a grouped spatial multi-scale convolution module and a feature enhancement fusion network to improve the YOLOv7 target detection model, obtain an improved pre-trained defect detection model, obtain a data set of rolled steel surface defect images to train the pre-trained defect detection model to obtain a converged defect detection model, input the required detection image into the defect detection model, and obtain an image output by the defect detection model with the defect location and defect type marked, thereby improving the expression ability of image extraction features and greatly increasing the defect detection performance of the model.

[0007] The method for detecting surface defects of rolled steel based on an improved group convolution and feature enhancement fusion network based on a multi-head attention mechanism provided by an embodiment of the present invention comprises:

[0008] Acquire a dataset of steel rolling surface defect images;

[0009] Based on the YOLOv7 target detection model, grouped spatial multi-scale convolution module and feature enhancement fusion network, the defect detection model is trained according to the steel rolling surface defect image dataset;

[0010] Obtain the required detection image;

[0011] The desired inspection image is input into the defect detection model to obtain the target rolled steel surface defect detection result in the desired inspection image.

[0012] Preferably, the grouped space multi-scale convolution module is constructed as follows:

[0013] Based on the multi-head self-attention mechanism, grouped spatial multi-scale convolution and the feed-forward network with gating mechanism, a grouped spatial multi-scale convolution module is constructed.

[0014] Preferably, the feature enhancement fusion network is constructed as follows:

[0015] The outputs of the grouped spatial multi-scale convolutional modules are cross-scale connected and multi-scale feature fused to obtain a feature enhanced fusion network.

[0016] Preferably, the feature enhancement fusion network further includes:

[0017] Use GELU activation function to replace the ReLU activation function of the YOLOv7 target detection model;

[0018] The improved MPD I oU loss function is used to optimize the detection results.

[0019] The steel rolling surface defect detection method based on the improved group convolution and feature enhancement fusion network of the multi-head attention mechanism provided by the embodiment of the present invention also includes:

[0020] Add a small-scale target detection layer to the detection head module of the YOLOv7 target detection model.

[0021] Preferably, obtaining a steel rolling surface defect image dataset includes:

[0022] Obtain local steel rolling surface defect image dataset;

[0023] According to the local steel rolling surface defect image dataset, a local defect type set is obtained;

[0024] If the local defect type set satisfies the standard defect type set, each local defect type in the local defect type set is traversed in turn, and the local defect type being traversed is used as the target standard defect type;

[0025] Acquire a first image feature set of local defect image data of a target standard defect type;

[0026] Acquire a preset second image feature set corresponding to a standard defect type;

[0027] Performing image feature matching on the first image feature set and the second image feature set, and determining that the second image feature set that does not have a corresponding match is the third image feature set;

[0028] After all local defect types are traversed, obtain the index building template;

[0029] Building a template based on the index, building a first target index according to the third image feature set and the corresponding standard defect type;

[0030] If the local defect type set does not satisfy the standard defect type set, the missing standard defect type in the local defect type set is determined and used as the missing defect type;

[0031] Building a template based on the index, and building a second target index according to the missing defect type;

[0032] Based on the big data expansion node, according to the first target index and the second target index, a supplementary steel rolling surface defect image dataset is obtained.

[0033] The local steel rolling surface defect image dataset and the supplementary steel rolling surface defect image dataset are collectively used as the steel rolling surface defect image dataset.

[0034] Preferably, obtaining the required detection image includes:

[0035] Obtaining the required detection image taken by the shooting node based on the preset shooting rules;

[0036] and / or,

[0037] Acquire the shooting trigger feature, and trigger the shooting node to acquire the required detection image based on the shooting trigger feature.

[0038] Preferably, obtaining the shooting trigger feature includes:

[0039] Obtaining records of manual steel rolling defect detection process;

[0040] The manual steel rolling defect detection process is recorded on a preset time axis to obtain multiple manual defect detection contents and detection time points; the manual defect detection contents are: manual steel rolling defect detection process record items;

[0041] Parse the artificial defect detection content and obtain the detection content semantic set;

[0042] If any detection content semantics in the detection content semantic set has a semantic match with the preset trigger verification semantics, obtaining the parsed source content of the detection content semantics with the corresponding semantic match;

[0043] Obtain the artificial defect detection content after parsing the source content on the timeline and use it as the attribution content;

[0044] Parse the attributed content and obtain the attributed content semantic set;

[0045] Determine the suspicious steel rolling process description semantics according to the attributed content semantic set and the preset steel rolling process description semantic library;

[0046] Obtain the number of suspicious steel rolling processes described by the semantic description of the suspicious steel rolling process;

[0047] If the number of suspicious steel rolling processes is 1, the attributed content semantic set where the suspicious steel rolling process description semantics is located is used as the shooting trigger feature;

[0048] If the number of suspicious steel rolling processes is greater than 1, an attempt is made to obtain the negative semantics in the attribution content semantic set where the description semantics of the suspicious steel rolling process are located. If the attempt fails, the corresponding attribution content semantic set is used as the shooting trigger feature.

[0049] Preferably, triggering the shooting node to acquire the required detection image based on the shooting trigger feature includes:

[0050] Acquire process monitoring images of steel rolling process;

[0051] According to the process monitoring image, the monitoring content description semantics is obtained and used as the feature to be matched;

[0052] Perform feature matching between the shooting trigger feature and the feature to be matched. If the match is satisfactory, obtain the steel rolling production identification according to the corresponding process monitoring image;

[0053] The required inspection images are captured based on the steel rolling production identification and the camera preset in the steel rolling production inspection terminal.

[0054] Preferably, obtaining the monitoring content description semantics according to the process monitoring image includes:

[0055] Extracting image features from the process monitoring image to obtain process monitoring image features;

[0056] Get the image description model;

[0057] The process monitoring image features are input into the image description model to obtain the monitoring content description semantics.

[0058] The steel rolling surface defect detection system based on the improved group convolution and feature enhancement fusion network of the multi-head attention mechanism provided by the embodiment of the present invention comprises:

[0059] The steel rolling surface defect data acquisition subsystem is used to acquire the steel rolling surface defect image data set;

[0060] The defect detection model training subsystem is used to train the defect detection model based on the steel rolling surface defect image dataset based on the YOLOv7 target detection model, the grouped space multi-scale convolution module and the feature enhancement fusion network;

[0061] A required detection image acquisition subsystem is used to acquire the required detection image;

[0062] The defect detection subsystem is used to input the required detection image into the defect detection model to obtain the target rolled steel surface defect detection result in the required detection image.

[0063] The beneficial effects of the present invention are:

[0064] The present invention introduces a grouped space multi-scale convolution module and a feature enhancement fusion network to improve the YOLOv7 target detection model, obtains an improved pre-trained defect detection model, obtains a steel rolling surface defect image data set to train the pre-trained defect detection model to obtain a converged defect detection model, inputs the required detection image into the defect detection model, obtains an image output by the defect detection model with the defect location and defect type marked, improves the expression ability of image extraction features, and greatly improves the model defect detection performance.

[0065] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in the present application documents.

[0066] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0068] Figure 1 It is a schematic diagram of a method for detecting surface defects of rolled steel based on an improved group convolution and feature enhancement fusion network of a multi-head attention mechanism in an embodiment of the present invention;

[0069] Figure 2 is a schematic diagram of a CFB module in an embodiment of the present invention;

[0070] Figure 3 is a schematic diagram of a feature enhancement fusion network in an embodiment of the present invention;

[0071] Figure 4 It is a schematic diagram of a steel rolling surface defect detection system based on an improved group convolution and feature enhancement fusion network of a multi-head attention mechanism in an embodiment of the present invention. DETAILED DESCRIPTION

[0072] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0073] The embodiment of the present invention provides a method for detecting surface defects of rolled steel based on an improved group convolution and feature enhancement fusion network based on a multi-head attention mechanism, such as Figure 1 As shown, including:

[0074] Step 1: Acquire a steel rolling surface defect image dataset; wherein the steel rolling surface defect image dataset is a dataset containing a plurality of image data labeled with defects on the steel rolling surface (such as rolling scale, plaque, crack, pitting surface, inclusions and scratches);

[0075] Step 2: Based on the YOLOv7 target detection model, the grouped spatial multi-scale convolution module and the feature enhancement fusion network, the defect detection model is trained according to the steel rolling surface defect image dataset; among them, the grouped spatial multi-scale convolution (CFB) module is as follows Figure 2 As shown in , it consists of a multi-head self-attention mechanism, grouped spatial multi-scale convolution, and a feedforward network based on a gating mechanism; the feature enhancement fusion network, such as Figure 3As shown, specifically: a neck network of cross-scale connection and multi-scale feature fusion; during training, the original backbone network of the YOLOv7 target detection model is replaced by a grouped spatial multi-scale convolution module, and the original neck network of the YOLOv7 target detection model is replaced by a feature enhancement fusion network. After the replacement is completed, a pre-trained defect detection model is obtained, and the steel surface defect image data in the steel surface defect image data set is used as the input of the pre-trained defect detection model, and the defect recognition result annotated by the steel surface defect image data is used as the output of the pre-trained defect detection model to train the pre-trained defect detection model, and a defect detection model that reaches an artificially preset convergence standard is obtained;

[0076] Step 3: Obtain the required inspection image; wherein the required inspection image is: an image required for steel rolling surface defect inspection, and the required inspection image can be input manually or obtained by a camera at the terminal of the steel rolling production line;

[0077] Step 4: Input the required inspection image into the defect detection model to obtain the target rolled steel surface defect detection result in the required inspection image. The target rolled steel surface defect detection result is: the required inspection image with the rolled steel defect area and defect type marked.

[0078] The working principle and beneficial effects of the above technical solution are:

[0079] The present invention uses the NEU-DET dataset released by Northeastern University for experiments. The NEU-DET dataset contains an image collection of six common defect types on the surface of rolled steel, comprehensively covering typical surface problems in the steel rolling production process. The dataset contains 1,800 grayscale images, 300 of each defect type.

[0080] In view of the problem of insufficient feature extraction capability in the YOLOv7 backbone network, this embodiment introduces a grouped spatial multi-scale convolution (CFB) module. The CFB module consists of a multi-head self-attention mechanism, a grouped spatial multi-scale convolution, and a feedforward network based on a gating mechanism. Compared with the traditional convolution operation, the multi-scale convolution module uses a multi-scale large convolution kernel attention mechanism to split the input vector along the channel dimension. One group concatenates the result of convolution of a 3×3 convolution layer and a 5×5 dilated convolution layer and the result of convolution of a 5×5 convolution layer and a 7×7 dilated convolution layer to obtain an enhanced feature map. For different receptive fields, it can capture multi-scale context information and enhance language information. After multiplying the enhanced feature map with another group of input features, the dimension is reduced through a convolution operation, and a jump connection is made with the initial input features to maintain gradient flow and enhance feature representation capability.

[0081] In the initialization stage of the multi-head attention mechanism, the model needs to set the weights of the query, key, and value matrices and the number of heads. The initialization of these weights is to ensure that the model can effectively learn the relationship between the input features during the training process; next, the input query, key, and value matrices will be linearly transformed. By multiplying with the corresponding weight matrix, these input features are converted into a format that can be used for attention calculation. This step is crucial for extracting relevant information about the input features to ensure that subsequent attention calculations can be performed effectively. After completing the linear transformation, the query, key, and value matrices will be divided into multiple heads, each with independent query, key, and value matrices. This design allows the model to calculate attention scores in parallel in different subspaces, enhancing the diversity of feature extraction. Different attention heads can focus on different features of the input data, thereby capturing richer information; subsequently, the attention scores of each head are scaled to avoid the problem of gradient disappearance or explosion during the calculation process; then, the Softmax function is applied The attention scores are normalized so that the sum of the weights of each position is 1, which ensures the stability and interpretability of the attention mechanism. After normalization, the model uses these attention weights to perform weighted summation on the value matrix to obtain the output matrix of each head. In this way, the model can obtain diverse information from different heads, further enriching the feature expression. Finally, the output matrices of all heads are spliced ​​together to form a large output matrix, and after a linear transformation, the final output is obtained. This process enables the model to integrate the information extracted by each attention head to obtain a comprehensive understanding of the input data. In the application of steel surface defect detection, the use of a multi-head attention mechanism enables the model to better pay attention to defects of different scales and features, improves the robustness and accuracy of detection, and solves the problem of insufficient detection accuracy of traditional methods in complex backgrounds. By integrating multi-faceted information, the model can effectively capture subtle defect features and significantly enhance detection capabilities.

[0082] In the process of grouped convolution, the multi-head self-attention mechanism is introduced to enable the model to establish stronger global associations between different spatial positions, thereby improving the recognition of defect patterns. At the same time, grouped convolution can also improve the parallelism of the model, so that the module can run efficiently under a larger input scale. The module implements multi-scale feature extraction, which can better capture the relationship between details and global patterns in the image, and is suitable for processing defects of various scales on the steel surface, such as tiny scratches or larger dents.

[0083] The feedforward network module of the gating mechanism expands the number of channels of the input feature map by two times and then processes it through the deep convolution layer, and continues to expand the number of output channels. Each channel is independently convolved, thereby retaining more local spatial information and reducing computational complexity; the output feature map after the deep convolution layer is divided into two parts along the channel dimension, one part is nonlinearly transformed by the Ge lu activation function and then multiplied element by element with the other part to realize the gating mechanism, and the gating mechanism is used to selectively retain important features; through the deep convolution and gating mechanism, the overall computational complexity is not significantly increased, and important features can be better selected, which improves the robustness of the model to noise and irrelevant information, enables it to focus more accurately on defective areas, and enhances the fine-grained feature extraction capability of the model, which is very effective for detecting tiny flaws or irregular areas;

[0084] In summary, the CFB module can effectively improve the feature extraction ability and robustness of the model through the combination of multi-scale convolution, self-attention and gating mechanism, and is particularly suitable for detecting steel surface defects in complex backgrounds.

[0085] The embodiment of the present invention also constructs a neck network with cross-scale connection and multi-scale feature fusion to enhance the feature fusion capability; the architecture particularly emphasizes the importance of the early feature layer, effectively integrating the rich underlying semantic information into the final feature fusion process, so as to give full play to its role in capturing detail features; different from the three-dimensional input feature map of the original YOLOv7 neck network, a large-scale and low-level feature input is additionally introduced, so that the model can more comprehensively utilize the early feature layer to maximize the retention of texture and edge information in the image; in addition, the feature enhancement fusion architecture not only fuses the feature maps of the same level, but also fuses the feature maps of different dimensions from the backbone network through the cross-scale connection of the feature maps. This strategy not only strengthens the deep feature mapping, but also ensures that the underlying detail features that are particularly critical for detection are retained; the feature fusion architecture using early intervention and multi-dimensional cross-scale connection strategies introduces rich underlying contextual information into the model, greatly enhancing the model's ability to characterize deep-level abstract features during the learning process; significantly improves the performance of the model in capturing difficult-to-detect defects such as fine cracks and slight flaws, and brings significant performance improvements to the task of steel rolling surface defect detection.

[0086] The present invention introduces a grouped space multi-scale convolution module and a feature enhancement fusion network to improve the YOLOv7 target detection model, obtains an improved pre-trained defect detection model, obtains a steel rolling surface defect image data set to train the pre-trained defect detection model to obtain a converged defect detection model, inputs the required detection image into the defect detection model, obtains an image output by the defect detection model with the defect location and defect type marked, improves the expression ability of image extraction features, and greatly improves the model defect detection performance.

[0087] In one embodiment, the feature enhancement fusion network further includes:

[0088] Use GELU activation function to replace the ReLU activation function of the YOLOv7 target detection model;

[0089] The improved MPD I oU loss function is used to optimize the detection results.

[0090] The working principle and beneficial effects of the above technical solution are:

[0091] The feature enhancement fusion network uses the GELU activation function to replace the ReLU activation function of YOLOv7 to improve nonlinear expression capabilities, and uses the improved MPD I oU loss function to optimize the detection results and improve the bounding box positioning accuracy. Compared with the traditional ReLU activation function, GELU can smoothly process the activation value, making the model more sensitive to small numerical fluctuations, which can improve the stability and expressiveness of the model. The improved MPD I oU loss function is more accurate than the traditional IoU loss in locating defect boundaries, and can reduce boundary errors, especially when detecting defects whose boundaries are difficult to accurately define, showing better results.

[0092] The MPDI oU loss calculation principle is to directly calculate the distance between the upper left corner and the lower right corner of the minimized predicted bounding box and the actual labeled bounding box. The formula for the distance d1 and d2 between two points in the similarity matrix is ​​as follows:

[0093]

[0094] are the left vertex coordinates and the lower right point coordinates of point A; are the left vertex coordinates and the lower right point coordinates of point B. The MPD I oU loss function calculation formula is as follows:

[0095]

[0096] The GELU activation function is a nonlinear activation function based on Gaussian distribution. It combines the non-saturation characteristics of the ReLU activation function and the smoothness characteristics of the Sigmad activation function. This smoothness is more stable when the neural network processes discontinuous input data, and it can alleviate the problem of gradient disappearance when training deep networks.

[0097] The mathematical expression of the GELU activation function is as follows:

[0098] GELU(x)=x·φ(x)

[0099]

[0100] Where φ(x) is the standard normal cumulative distribution function of the input x, and erf(x) is the error function used to calculate the Gaussian integral of x. The final formula is as follows:

[0101]

[0102] The feature enhancement fusion network of the present invention combines the MPD I oU loss function and the GELU activation function to improve the accuracy of feature matching and the training stability.

[0103] The embodiment of the present invention provides a method for detecting surface defects of rolled steel based on an improved group convolution and feature enhancement fusion network based on a multi-head attention mechanism, and further includes:

[0104] Add a small-scale target detection layer to the detection head module of the YOLOv7 target detection model.

[0105] The working principle and beneficial effects of the above technical solution are:

[0106] The defect information localization module adds a detection layer specifically for processing small-scale targets in the detection head module to enhance the model's ability to finely identify defects of different sizes, especially small defects. This detection layer uses an architecture designed for small targets, allowing the network to better retain important details of small targets during feature extraction. In the original detection head structure, detection is usually mainly performed on medium or large-sized targets, and the newly added small target detection layer further enhances the model's performance in processing small-sized defects, thereby introducing the ability to process multi-scale features in the overall detection framework.

[0107] The new detection layer in the present invention enhances the ability to capture tiny defects and strengthens the image detail information through the receptive field expansion mechanism. The model can process defect information of different scales at the same time, achieving comprehensive coverage of multi-scale targets.

[0108] In one embodiment, obtaining a steel rolling surface defect image dataset includes:

[0109] Acquire a local steel rolling surface defect image dataset; wherein the local steel rolling surface defect image dataset is: a dataset containing local steel rolling surface defect images, and the local steel rolling surface defect images are steel rolling surface defect images recorded locally by the defect detection party, such as: NEU-DET dataset;

[0110] According to the local steel rolling surface defect image data set, a local defect type set is obtained; wherein the local defect type set is: a set of defect types that have been monitored locally, such as rolling scale, plaque, crack, pitting surface, inclusions and scratches;

[0111] If the local defect type set satisfies the standard defect type set, each local defect type in the local defect type set is traversed in turn, and the local defect type being traversed is used as the target standard defect type; wherein the standard defect type set is: a collection of all defect types retrieved based on the knowledge graph of the steel rolling industry; the local defect type set satisfies the standard defect type set means that all standard defect types have been recorded locally;

[0112] Acquire a first image feature set of local defect image data of a target standard defect type; wherein the first image feature set is: a feature representation of image data of the local defect image data of the target standard defect type, such as: image color, image texture, etc. when the target standard defect type extracted from the local defect image data is a pitting surface;

[0113] Obtain a preset second image feature set corresponding to the standard defect type; wherein the second image feature set is: a set of feature-represented defect image data of the standard defect type obtained by big data, and the second image feature is extracted in the same manner as the first image feature;

[0114] Performing image feature matching on the first image feature set and the second image feature set, and determining that the second image feature set that does not have a corresponding match is the third image feature set;

[0115] When all local defect types are traversed, an index building template is obtained; wherein the index building template is a template for generating index conditions by comparing index condition generation information and generating a big data index based on the index conditions, and the index condition generation information includes: defect type, or defect type and image feature set corresponding to the defect type;

[0116] Building a template based on the index, building a first target index according to the third image feature set and the corresponding standard defect type;

[0117] If the local defect type set does not satisfy the standard defect type set, the missing standard defect type in the local defect type set is determined and used as the missing defect type;

[0118] Based on the index construction template, a second target index is constructed according to the missing defect type; wherein the construction rule of the second target index is the same as the first target index;

[0119] Based on the big data expansion node, a supplementary steel rolling surface defect image data set is obtained according to the first target index and the second target index;

[0120] The local steel rolling surface defect image dataset and the supplementary steel rolling surface defect image dataset are collectively used as the steel rolling surface defect image dataset.

[0121] The working principle and beneficial effects of the above technical solution are:

[0122] When the present invention uses the steel rolling surface defect image data set to train the defect detection model, the local steel rolling surface defect image data set (a collection of steel rolling surface defect image data that has been collected locally) is first obtained, and a local defect type set is extracted. At the same time, a standard defect type set is introduced (a collection of all defect types retrieved based on the knowledge graph of the steel rolling industry). According to the local defect type set and the standard defect type set, it is judged whether the local defect type is comprehensive. If it is comprehensive, each local defect type is traversed, and the first image feature set of the local defect image data of the target standard defect type being traversed and the second image feature set preset for the standard defect type are extracted. The first image feature set and the second image feature set are matched for image features to determine the defect images that are missing locally and contain the third image feature set; all local defect types are traversed. After the completion of the process, a template is constructed based on the index, and a first target index of the defect image containing the third image feature set is constructed according to the third image feature set and the corresponding standard defect type; if the local defect type set does not meet the standard defect type set, it means that the defect types contained in the local image are incomplete, then a template is directly constructed based on the index, and a second target index is constructed according to the missing defect type, and all defect images of the missing defect type are indexed, and based on the big data expansion node, a supplementary steel rolling surface defect image dataset is obtained according to the first target index and the second target index, and the supplementary steel rolling surface defect image dataset is supplemented in a targeted manner, and the local steel rolling surface defect image dataset and the supplementary steel rolling surface defect image dataset are used together as the steel rolling surface defect image dataset, which improves the comprehensiveness of the training data and the quality of subsequent model training.

[0123] In one embodiment, obtaining a desired detection image includes:

[0124] Obtain the required inspection image captured by the shooting node based on the preset shooting rule; wherein the shooting node is: a network node of a camera on the steel rolling inspection side, and the preset shooting rule is manually pre-set, such as: shooting at a fixed time period;

[0125] and / or,

[0126] Acquire the shooting trigger feature, and trigger the shooting node to acquire the required inspection image based on the shooting trigger feature. The shooting trigger feature is the feature that triggers the camera shooting on the steel rolling inspection side. For example, the standardization of the steel rolling process will affect the quality of the steel rolling, and the non-standard description feature of the steel rolling process is used as the shooting trigger feature.

[0127] The working principle and beneficial effects of the above technical solution are:

[0128] The present invention introduces two methods to obtain the required detection images; the first method: manually preset shooting rules to shoot the required detection images; the second method: introduce shooting trigger features to adaptively acquire the required detection images, thereby reducing subsequent detection computing power and reducing energy consumption.

[0129] In one embodiment, obtaining a shooting trigger feature includes:

[0130] Obtaining a process record of manual steel rolling defect detection; wherein the process record of manual steel rolling defect detection is: a process record of manual steel rolling defect detection;

[0131] The manual steel rolling defect detection process is recorded and expanded on a preset time axis to obtain multiple manual defect detection contents and detection time points; the manual defect detection contents are: manual steel rolling defect detection process record items, such as: detection of strip head folding conditions record items; wherein the manual defect detection contents and detection time points correspond one to one;

[0132] Parse the artificial defect detection content and obtain the detection content semantic set; wherein the detection content semantic set is; the description semantics of the detection content, such as: "this piece of rolled steel is cracked", "this piece of rolled steel has scratches";

[0133] If any detection content semantics in the detection content semantic set has a semantic match with the preset trigger verification semantics, the parsed source content of the detection content semantics with the corresponding semantic match is obtained; wherein the preset trigger verification semantics is manually preset, specifically the defect description semantics describing the defects on the surface of the rolled steel; the parsed source content is the artificial defect detection content extracted corresponding to the detection content semantics;

[0134] Obtain the artificial defect detection content after parsing the source content on the timeline and use it as the attribution content;

[0135] Parse the attributed content and obtain the attributed content semantic set; wherein the attributed content and the attributed content semantic set correspond one to one, and the attributed content semantics are, for example, "the strip head flies up when the coiling pinch roller is threaded", "the rust falls onto the surface of the slab".

[0136] Determine the suspicious steel rolling process description semantics according to the attributed content semantic set and the preset steel rolling process description semantic library; wherein the preset steel rolling process description semantic library includes: description semantics of different steel rolling processes (such as heating, rough rolling, finishing rolling, cooling, etc.);

[0137] Obtain the number of suspicious steel rolling processes described by the semantic description of the suspicious steel rolling process;

[0138] If the number of suspicious steel rolling processes is 1, the attributed content semantic set where the suspicious steel rolling process description semantics is located is used as the shooting trigger feature;

[0139] If the number of suspicious steel rolling processes is greater than 1, then try to obtain the negative semantics in the attributed content semantic set where the suspicious steel rolling process description semantics are located. If the attempt to obtain fails, use the corresponding attributed content semantic set as the shooting trigger feature;

[0140] The method of triggering the shooting node to obtain the required detection image based on the shooting trigger feature includes:

[0141] Acquire process monitoring images of the steel rolling process; wherein the process monitoring images are monitoring images of different steel rolling processes, such as images of the heating process of the steel billet and images of the heated steel billet entering the rough rolling mill for rough rolling;

[0142] According to the process monitoring image, the monitoring content description semantics is obtained and used as the feature to be matched; the monitoring content description semantics is, for example, "rust falls onto the slab surface";

[0143] The shooting trigger feature and the feature to be matched are feature matched, and if the match is met, the steel rolling production identification is obtained according to the corresponding process monitoring image; wherein the steel rolling production identification is: when the shooting trigger feature and the feature to be matched match, the unique identification of the steel rolling piece being produced in the corresponding process monitoring image;

[0144] Based on the steel rolling production mark and the camera preset in the steel rolling production inspection terminal, the required inspection image is captured; wherein, when the required inspection image is captured, the shooting node recognizes the steel rolling production mark and automatically captures the surface image of the steel rolling part with the corresponding mark;

[0145] Among them, according to the process monitoring image, the monitoring content description semantics is obtained, including:

[0146] Extracting image features from the process monitoring image to obtain process monitoring image features; wherein the process monitoring image features include: equipment operation of the operating equipment in the process monitoring image, the shape and color of the steel billet, the behavior of the operator, and the movement of other objects in the image (e.g., rusted objects dropped due to rolling mill vibration, etc.);

[0147] Obtain an image description model; wherein the image description model is: an AI model that describes an image based on image features, using multiple known image features as inputs of a CNN model and image descriptions corresponding to the known image features as outputs of the CNN model for training and acquisition;

[0148] The process monitoring image features are input into the image description model to obtain the monitoring content description semantics.

[0149] The working principle and beneficial effects of the above technical solution are:

[0150] When the present invention obtains the trigger feature of the shooting node, the artificial steel rolling defect detection process record is introduced, the artificial steel rolling defect detection process record is expanded on the time axis, and a plurality of one-to-one corresponding artificial steel rolling defect detection process record items and detection time points are obtained. The detection content semantic set of the artificial defect detection content is obtained based on the semantic analysis technology, and the detection content semantics in the detection content semantic set are semantically matched with the preset trigger verification semantics. If there is a semantic description of defects on the steel rolling surface, the corresponding extracted artificial defect detection content is used as the parsed source content; the attribution content after the parsed source content on the time axis is obtained, and the attribution content semantic set is obtained; the steel rolling is introduced The process description semantic library determines the suspicious steel-rolling process extracted from the attribution content and determines the number of suspicious steel-rolling processes; if the number of suspicious steel-rolling processes is only 1, it can be directly determined that the attribution content semantic set where the corresponding suspicious steel-rolling process description semantics is located is for attribution description of steel-rolling defects, so it is used as a shooting trigger feature; if the number of suspicious steel-rolling processes is greater than 1, it is possible that there are indeed multiple irregular steel-rolling processes, or it is possible that it includes suspicious steel-rolling processes that are excluded after discussion. Therefore, try to obtain the negative semantics in the attribution content semantic set where the suspicious steel-rolling process description semantics is located. If there is no negative semantics, the corresponding attribution content semantic set is used as a shooting trigger feature.

[0151] When the shooting node is triggered to obtain the required detection image based on the shooting trigger feature, it is necessary to match the description situation. Therefore, the process monitoring image of the steel rolling process is obtained, and the monitoring content description semantics of the process monitoring image is used as the feature to be matched. The monitoring content description semantics of the process monitoring image is obtained by inputting the process monitoring image features extracted from the process monitoring image into the image description model. After acquisition, the shooting trigger feature and the feature to be matched are feature matched. If the match is met, it means that the corresponding production has a high probability of steel rolling defects. Determine its production identification, identify its identification and shoot it at the terminal for subsequent defect detection, which greatly improves the detection efficiency of defective parts.

[0152] The embodiment of the present invention provides a steel rolling surface defect detection system based on an improved group convolution and feature enhancement fusion network with a multi-head attention mechanism, such as Figure 4 As shown, including:

[0153] The steel rolling surface defect data acquisition subsystem 1 is used to acquire a steel rolling surface defect image data set;

[0154] Defect detection model training subsystem 2, used to train a defect detection model based on a YOLOv7 target detection model, a grouped spatial multi-scale convolution module, and a feature enhancement fusion network according to a steel rolling surface defect image dataset;

[0155] The required detection image acquisition subsystem 3 is used to acquire the required detection image;

[0156] The defect detection subsystem 4 is used to input the required detection image into the defect detection model to obtain the target rolled steel surface defect detection result in the required detection image.

[0157] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for detecting surface defects of rolled steel based on an improved group convolution and feature enhancement fusion network with multi-head attention mechanism, characterized in that: include: Acquire a dataset of steel rolling surface defect images; Based on the YOLOv7 target detection model, grouped spatial multi-scale convolution module and feature enhancement fusion network, the defect detection model is trained according to the steel rolling surface defect image dataset; Obtain the required detection image; The desired inspection image is input into the defect detection model to obtain the target rolled steel surface defect detection result in the desired inspection image.

2. The method for detecting surface defects of rolled steel based on an improved group convolution and feature enhancement fusion network with multi-head attention mechanism as claimed in claim 1, characterized in that: The grouped spatial multi-scale convolution module is constructed as follows: Based on the multi-head self-attention mechanism, grouped spatial multi-scale convolution and the feed-forward network with gating mechanism, a grouped spatial multi-scale convolution module is constructed.

3. The method for detecting surface defects of rolled steel based on an improved group convolution and feature enhancement fusion network based on a multi-head attention mechanism as claimed in claim 1, characterized in that: The feature enhancement fusion network is constructed as follows: The outputs of the grouped spatial multi-scale convolutional modules are cross-scale connected and multi-scale feature fused to obtain a feature enhanced fusion network.

4. The method for detecting surface defects of rolled steel based on an improved group convolution and feature enhancement fusion network with multi-head attention mechanism as claimed in claim 1, characterized in that: Feature enhancement fusion network also includes: Use GELU activation function to replace the ReLU activation function of the YOLOv7 target detection model; The improved MPDIoU loss function is used to optimize the detection results.

5. The method for detecting surface defects of rolled steel based on improved group convolution and feature enhancement fusion network with multi-head attention mechanism as claimed in claim 1, characterized in that: Also includes: Add a small-scale target detection layer to the detection head module of the YOLOv7 target detection model.

6. The method for detecting surface defects of rolled steel based on an improved group convolution and feature enhancement fusion network with multi-head attention mechanism as claimed in claim 1, characterized in that: Obtain the required detection images, including: Obtaining the required detection image taken by the shooting node based on the preset shooting rules; and / or, Acquire the shooting trigger feature, and trigger the shooting node to acquire the required detection image based on the shooting trigger feature.

7. The method for detecting surface defects of rolled steel based on an improved group convolution and feature enhancement fusion network with multi-head attention mechanism as claimed in claim 6, characterized in that: Get the shooting trigger characteristics, including: Obtaining records of manual steel rolling defect detection process; The manual steel rolling defect detection process is recorded on a preset time axis to obtain multiple manual defect detection contents and detection time points; the manual defect detection contents are: manual steel rolling defect detection process record items; Parse the artificial defect detection content and obtain the detection content semantic set; If any detection content semantics in the detection content semantic set has a semantic match with the preset trigger verification semantics, obtaining the parsed source content of the detection content semantics with the corresponding semantic match; Obtain the artificial defect detection content after parsing the source content on the timeline and use it as the attribution content; Parse the attributed content and obtain the attributed content semantic set; Determine the suspicious steel rolling process description semantics according to the attributed content semantic set and the preset steel rolling process description semantic library; Obtain the number of suspicious steel rolling processes described by the semantic description of the suspicious steel rolling process; If the number of suspicious steel rolling processes is 1, the attributed content semantic set where the suspicious steel rolling process description semantics is located is used as the shooting trigger feature; If the number of suspicious steel rolling processes is greater than 1, an attempt is made to obtain the negative semantics in the attributed content semantic set where the description semantics of the suspicious steel rolling process are located. If the attempt fails, the corresponding attributed content semantic set is used as the shooting trigger feature.

8. The method for detecting surface defects of rolled steel based on improved group convolution and feature enhancement fusion network with multi-head attention mechanism as claimed in claim 6, characterized in that: Based on the shooting trigger feature, the shooting node is triggered to obtain the required detection image, including: Acquire process monitoring images of steel rolling process; According to the process monitoring image, the monitoring content description semantics is obtained and used as the feature to be matched; Perform feature matching between the shooting trigger feature and the feature to be matched. If the match is satisfactory, obtain the steel rolling production identification according to the corresponding process monitoring image; The required inspection images are captured based on the steel rolling production identification and the camera preset in the steel rolling production inspection terminal.

9. The method for detecting surface defects of rolled steel based on improved group convolution and feature enhancement fusion network with multi-head attention mechanism as claimed in claim 8, characterized in that: According to the process monitoring image, obtain the monitoring content description semantics, including: Extracting image features from the process monitoring image to obtain process monitoring image features; Get the image description model; The process monitoring image features are input into the image description model to obtain the monitoring content description semantics.

10. A steel rolling surface defect detection system based on an improved group convolution and feature enhancement fusion network with multi-head attention mechanism, characterized in that: include: The steel rolling surface defect data acquisition subsystem is used to acquire the steel rolling surface defect image data set; The defect detection model training subsystem is used to train the defect detection model based on the steel rolling surface defect image dataset based on the YOLOv7 target detection model, the grouped space multi-scale convolution module and the feature enhancement fusion network; A required detection image acquisition subsystem is used to acquire the required detection image; The defect detection subsystem is used to input the required detection image into the defect detection model to obtain the target rolled steel surface defect detection result in the required detection image.

Images