Drum net surface defect detection method, system, equipment and medium

Abstract

The invention relates to the field of image detection, in particular to a drum net surface defect detection method, system and device and a medium, and the method comprises the steps: obtaining a to-be-detected drum net surface image, carrying out the input adaption processing of the to-be-detected drum net surface image, and generating a net surface image feature map; inputting the mesh image feature map into a mesh defect detection model for feature fusion enhancement to obtain an enhanced feature map; extracting feature maps with different resolutions from the enhanced feature map, and fusing the feature maps with different resolutions to generate a plurality of fused feature maps; and predicting a state category and a defect position of the mesh image feature map according to the fused feature map. According to the invention, fine defects can be effectively identified, and omission and misjudgment in manual inspection are reduced, so that the accuracy of defect detection is improved, the detection efficiency is greatly improved, and a guarantee is provided for timely discovery and repair of the defects.

Description

Technical Field

[0001] The present invention relates to the field of image detection, and particularly to a method, system, device and medium for detecting defects on the drum screen surface. Background Art

[0002] In a nuclear power plant, a drum screen is an important device specifically used to filter solid impurities in seawater. The surface characteristics of the drum screen are relatively unique. Due to its small mesh size and diverse and irregular defect types on the screen surface, it is challenging to effectively monitor it. Moreover, because the defect size on the drum screen surface is small and the background is complex, the difficulty of defect detection is further increased. Timely and effective detection of the defects on the drum screen can not only ensure the normal operation of the nuclear power plant but also avoid potential safety risks caused by drum screen defects, so it has important practical significance.

[0003] The existing methods for detecting drum screen defects mainly rely on manual inspection. This method not only has low detection efficiency but also is greatly affected by human factors, and the accuracy of its detection results cannot be guaranteed. In practical applications, manual inspection often results in missed detections and false detections, and cannot meet the high accuracy and high efficiency requirements of the nuclear power plant for drum screen defect detection. Summary of the Invention

[0004] To solve the above problems, the present invention provides a method, system, device and medium for detecting defects on the drum screen surface.

[0005] The first aspect of the present invention discloses a method for detecting defects on the drum screen surface, including:

[0006] Obtain the image of the drum screen surface to be detected, and perform input adaptation processing on it to generate a surface image feature map;

[0007] Input the surface image feature map into a surface defect detection model for feature fusion enhancement to obtain an enhanced feature map;

[0008] Extract feature maps with different resolutions from the enhanced feature map, and fuse the feature maps with different resolutions to generate multiple fused feature maps;

[0009] Predict the state category and defect location of the surface image feature map according to the fused feature map.

[0010] Further, the step of inputting the surface image feature map into a surface defect detection model for feature fusion enhancement to obtain an enhanced feature map includes:

[0011] Perform multiple non-linear transformations on the surface image feature map to generate a main branch feature map;

[0012] Perform dimensionality reduction on the mesh image feature map to generate an auxiliary branch feature map;

[0013] Perform channel dimension concatenation on the main branch feature map and the auxiliary branch feature map, and fuse the concatenation result with the mesh image feature map through a residual addition operation to generate an enhanced feature map.

[0014] Further, the step of generating the main branch feature map by performing multiple non-linear transformations on the mesh image feature map includes:

[0015] Use multiple cascaded bottleneck modules to perform multiple non-linear transformations on the mesh image feature map to generate the main branch feature map.

[0016] Further, the step of extracting feature maps with different resolutions from the enhanced feature map and fusing the feature maps with different resolutions to generate multiple fused feature maps includes:

[0017] Extract N feature maps with different resolutions from the enhanced feature map, and sort them in ascending order of resolution to obtain multiple sorted feature maps;

[0018] Traverse the first to the N-1th sorted feature maps:

[0019] Align the size of the nth sorted feature map with the (n + 1)th sorted feature map through upsampling operation to obtain an aligned feature map; where n ∈ [1, N-1];

[0020] Perform adaptive scaling on the aligned feature map to obtain a scaled and aligned feature map;

[0021] Add the scaled and aligned feature map and the (n + 1)th sorted feature map element by element to generate a fused feature map.

[0022] Further, the step of performing adaptive scaling on the aligned feature map to obtain a scaled and aligned feature map includes:

[0023] Determine the scale adjustment parameter corresponding to the sorted feature map according to the sorted feature map corresponding to the aligned feature map;

[0024] Scale the aligned feature map according to the scale adjustment parameter to obtain a scaled and aligned feature map.

[0025] Further, the step of determining the scale adjustment parameter corresponding to the sorted feature map according to the sorted feature map corresponding to the aligned feature map includes:

[0026] Determine the scaling ratio r of the (n + 1)th sorted feature map to the mesh image feature map n+1 ;

[0027] Determine the scale adjustment parameter s corresponding to the (n + 1)-th sorted feature map according to the following formula n+1 :

[0028] s n+1 = α·(1 - r n+1 ) + β;

[0029] where both α and β are preset hyperparameters.

[0030] Furthermore, the steps of predicting the state category and defect location of the mesh image feature map based on the fused feature map include:

[0031] Predict the class probability, center point offset, and bounding box of the fused feature map through an anchor-free prediction head;

[0032] Calculate the defect category of the drum mesh surface image to be detected according to the class probability, and calculate the defect location and size of the drum mesh surface image to be detected according to the center point offset and the bounding box.

[0033] Furthermore, the steps of predicting the class probability, center point offset, and bounding box of the fused feature map through an anchor-free prediction head include:

[0034] Predict the class probability of the fused feature map through an anchor-free prediction head according to the following formula Center point offset and bounding box

[0035]

[0036] where σ represents the Sigmoid activation function, W class is the class weight matrix, W1 is the class probability weight matrix, W2 is the center point offset weight matrix, W3 is the scale weight matrix, and F grid is the enhanced feature map extracted from the fused feature map.

[0037] Furthermore, the mesh defect detection model is trained according to a predefined loss function; where the loss function is the weighted sum of the distribution focal loss function and the smooth loss function.

[0038] Furthermore, the distribution focal loss function L df is:

[0039]

[0040] where I represents the number of mesh image feature maps in the training dataset, J represents the number of state categories, y oj represents the label of the i-th mesh image feature map corresponding to the j-th state category, and pij represents the probability that the i-th mesh image feature map belongs to the j-th state category, γ represents the focusing parameter, and α class represents the balance factor.

[0041] Furthermore, the smoothing loss function L smooth is:

[0042]

[0043] where x represents the offset of the predicted center point, the error between the bounding box and the ground truth, and W box represents the bounding box regression weight.

[0044] Furthermore, the steps of obtaining the drum mesh image to be detected and performing input adaptation processing on it to generate a mesh image feature map include:

[0045] Obtain the drum mesh image to be detected;

[0046] Perform grayscale conversion on the drum mesh image to be detected;

[0047] and / or perform feature scaling on the drum mesh image to be detected;

[0048] Generate a mesh image feature map.

[0049] Preferably, the steps of obtaining the drum mesh image to be detected and performing input adaptation processing on it to generate a mesh image feature map include:

[0050] Obtain the drum mesh image to be detected;

[0051] Perform histogram equalization processing and / or contrast enhancement processing on the drum mesh image to be detected;

[0052] Generate a mesh image feature map.

[0053] Preferably, the steps of obtaining the drum mesh image to be detected and performing input adaptation processing on it to generate a mesh image feature map include:

[0054] Obtain the drum mesh image to be detected;

[0055] Use an edge detection algorithm to extract edge features from the drum mesh image to be detected;

[0056] Generate a mesh image feature map.

[0057] Furthermore, the steps of obtaining the drum mesh image to be detected include:

[0058] Judge whether the size of the input drum mesh image exceeds a preset size threshold;

[0059] When it exceeds, the drum screen surface image is segmented into multiple images with sizes smaller than the size threshold, and multiple drum screen surface images to be detected are obtained.

[0060] The second aspect of the present invention discloses a drum screen surface defect detection system, including:

[0061] An acquisition module, configured to acquire a drum screen surface image to be detected, perform input adaptation processing on it, and generate a mesh surface image feature map;

[0062] An enhancement module, configured to input the mesh surface image feature map into a mesh surface defect detection model for feature fusion enhancement to obtain an enhanced feature map;

[0063] A fusion module, configured to extract feature maps with different resolutions from the enhanced feature map, fuse the feature maps with different resolutions, and generate multiple fused feature maps;

[0064] A prediction module, configured to predict the state category and defect position of the mesh surface image feature map according to the fused feature map.

[0065] The third aspect of the present invention discloses an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The feature is that when the processor executes the computer program, the steps of any of the drum screen surface defect detection methods disclosed in the first aspect of the present invention are implemented.

[0066] The fourth aspect of the present invention discloses a storage medium, which stores a computer program. The feature is that when the computer program is executed by a processor, the steps of any of the drum screen surface defect detection methods disclosed in the first aspect of the present invention are implemented.

[0067] Aiming at the deficiencies of the existing methods, the present invention proposes a drum screen surface defect detection method. Compared with the traditional manual inspection, the present invention can automatically analyze and predict the mesh surface image through the model, effectively identify subtle defects, reduce omissions and misjudgments in manual inspection, and thus improve the accuracy of defect detection; the automatic detection method can quickly process a large amount of mesh surface image data, realize real-time or near-real-time defect detection and identification, greatly improve the detection efficiency, and provide a guarantee for timely discovery and repair of defects. Description of the Drawings

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for description in the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0069] Figure 1 It is a schematic flowchart of a method for detecting defects on the drum mesh surface disclosed in an embodiment of the present invention;

[0070] Figure 2 It is a schematic structural diagram of a system for detecting defects on the drum mesh surface disclosed in an embodiment of the present invention;

[0071] Figure 3 It is a schematic structural diagram of an electronic device disclosed in an embodiment of the present invention. Detailed implementation manners

[0072] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0073] The terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, or product end that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, devices, or product ends.

[0074] Referring to "embodiment" herein means that a specific feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present invention. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0075] Please refer to Figure 1 as shown in Figure 1 It is a schematic flowchart of a method for detecting defects on the drum mesh surface disclosed in an embodiment of the present invention, including the following operations:

[0076] S101. Obtain an image of the drum mesh surface to be detected, perform input adaptation processing on it, and generate a feature map of the mesh surface image;

[0077] In an optional embodiment, the step of obtaining an image of the drum mesh surface to be detected and performing input adaptation processing on it to generate a feature map of the mesh surface image includes:

[0078] Obtain the image of the drum screen to be detected;

[0079] Perform grayscale conversion on the image of the drum screen to be detected;

[0080] And / or perform feature scaling on the image of the drum screen to be detected;

[0081] Generate a feature map of the screen image.

[0082] In this alternative embodiment, performing grayscale conversion and / or feature scaling on the image of the drum screen to be detected can reduce the influence of changes in lighting conditions on the detection result.

[0083] Grayscale conversion refers to converting a color image into a grayscale image. In a color image, each pixel is usually composed of the values of the red, green, and blue color channels, while in a grayscale image, each pixel has only one brightness value, representing different grayscale levels from black to white.

[0084] Grayscale conversion can simplify the color information of the image and reduce the computational complexity. For the drum screen defect detection task of the present invention, color information is not necessary and will instead increase the processing difficulty. By converting the image to grayscale, the focus can be placed on the brightness and texture features of the image, improving the efficiency and robustness of the analysis. Grayscale conversion can adopt methods such as the average value method and the weighted average method, and the embodiments of the present invention do not make limitations.

[0085] Feature scaling refers to adjusting the size of the image. Feature scaling can select bilinear interpolation, nearest neighbor interpolation, etc. Bilinear interpolation calculates the value of the new pixel by weighted averaging the image and can obtain a relatively smooth scaling effect. Nearest neighbor interpolation directly adopts the value of the nearest pixel, and the scaling effect is relatively sharp. In this embodiment, the size of the image of the drum screen to be detected is adjusted to 2024X2024 to improve the detection accuracy.

[0086] It can be seen that this alternative embodiment can reduce the influence of changes in lighting conditions on the detection result by performing grayscale conversion and feature scaling on the image of the drum screen to be detected. Grayscale conversion can eliminate the interference brought by color information and make the model more focused on the shape and texture features of the defects. Feature scaling can unify the screen images of different sizes to a size suitable for model input, improving the consistency and robustness of the detection. The combination of these two preprocessing steps can significantly improve the performance and generalization ability of the defect detection model and improve the accuracy and reliability of the detection.

[0087] In another alternative embodiment, the steps of obtaining the image of the drum screen to be detected and performing input adaptation processing on it to generate a feature map of the screen image include:

[0088] Obtain the image of the drum mesh surface to be detected;

[0089] Perform histogram equalization processing and / or contrast enhancement processing on the image of the drum mesh surface to be detected;

[0090] Generate a feature map of the mesh surface image.

[0091] In this alternative embodiment, histogram equalization processing refers to a method of enhancing image contrast by adjusting the grayscale histogram of the image. Specifically, the histogram equalization algorithm counts the frequency of each grayscale level in the image, and then maps the original grayscale levels to new grayscale levels according to the cumulative distribution function, so that the transformed grayscale histogram is as flat as possible. After histogram equalization processing, the brightness distribution of the image is more uniform, and the details in the dark and bright regions are enhanced.

[0092] Contrast enhancement processing is to stretch the grayscale range of the image, increase the difference between the bright and dark regions, thereby improving the overall contrast of the image. By non-linearly adjusting the pixel grayscale values, the dynamic range of the image is expanded, the bright areas become brighter, and the dark areas become darker, so that the image details are more obvious. The contrast enhancement method in this embodiment can use linear stretching, piecewise linear transformation, power-law transformation, logarithmic transformation, etc., and the embodiments of the present invention are not limited.

[0093] It can be seen that in this alternative embodiment, by performing histogram equalization processing and / or contrast enhancement processing on the image of the drum mesh surface to be detected, the quality and contrast of the image can be effectively improved, the characteristics of the mesh surface defects are highlighted, the difference between the defect region and the normal region is enhanced, and the subsequent feature extraction and defect detection are more accurate and reliable.

[0094] In another alternative embodiment, the steps of obtaining the image of the drum mesh surface to be detected and performing input adaptation processing on it to generate a feature map of the mesh surface image include:

[0095] Obtain the image of the drum mesh surface to be detected;

[0096] Use an edge detection algorithm to extract edge features from the image of the drum mesh surface to be detected;

[0097] Generate a feature map of the mesh surface image.

[0098] In this optional embodiment, the edge detection algorithm can extract the boundary information of the target object or region from a digital image. An edge is a region where the local features of an image change significantly, usually manifested as discontinuities in attributes such as grayscale, color, texture, etc. The edge detection algorithm identifies and locates these significantly changing pixel points or regions by performing mathematical operations and analyses on the image, such as gradient calculation, threshold processing, morphological operations, etc., so as to obtain the contour or segmentation result of the target object. The edge detection algorithm in the embodiments of the present invention can use the Canny edge detection algorithm, Sobel operator detection algorithm, Laplacian Operator detection algorithm, Roberts operator detection algorithm, etc., and the embodiments of the present invention are not limited thereto.

[0099] In yet another optional embodiment, the steps of obtaining the image of the drum screen to be detected include:

[0100] Determine whether the size of the input image of the drum screen exceeds a preset size threshold;

[0101] When it exceeds, the image of the drum screen is divided into multiple images with sizes smaller than the size threshold to obtain multiple images of the drum screen to be detected.

[0102] It can be seen that in this optional embodiment, by dividing a large image into multiple small blocks, the amount of data processed each time can be reduced, and the speed of feature extraction and defect detection can be accelerated. In addition, dividing a large image into multiple small blocks can also reduce the background complexity of a single region, reduce the influence of noise and interference on the detection result, and improve the accuracy of defect localization.

[0103] In an optional embodiment, the steps of obtaining the image of the drum screen to be detected and performing input adaptation processing on it to generate a feature map of the screen image include:

[0104] Perform normalization processing on the image of the drum screen to be detected to generate a feature map of the screen image.

[0105] In this optional embodiment, the image of the drum screen to be detected is normalized according to the following formula:

[0106]

[0107] where Img represents the image of the drum screen to be detected, min(Img) represents the minimum pixel value in the image of the drum screen to be detected, and max(Img) represents the maximum pixel value in the image of the drum screen to be detected.

[0108] It can be seen that in this optional embodiment, by performing normalization processing on the image of the drum screen to be detected, the brightness and contrast differences between different images can be eliminated, and the performance of model detection and recognition can be improved.

[0109] In a further optional embodiment, the Laplacian filter is used for the normalized drum mesh surface image to be detected, and the texture details of the image are enhanced according to the metal material characteristics of the mesh surface image.

[0110] It can be seen that in this optional embodiment, by applying the Laplacian filter to the normalized image, the texture information in the image can be further enhanced, making these features more prominent and easier to extract in subsequent feature extraction and analysis, thereby improving the performance of model detection and recognition.

[0111] S102. Input the mesh surface image feature map into a mesh surface defect detection model for feature fusion enhancement to obtain an enhanced feature map.

[0112] In an optional embodiment, the step of inputting the mesh surface image feature map into a mesh surface defect detection model for feature fusion enhancement to obtain an enhanced feature map includes:

[0113] Perform multiple non-linear transformations on the mesh surface image feature map to generate a main branch feature map.

[0114] Perform dimensionality reduction processing on the mesh surface image feature map to generate an auxiliary branch feature map.

[0115] Perform channel dimension splicing on the main branch feature map and the auxiliary branch feature map, and fuse the splicing result and the mesh surface image feature map through a residual addition operation to generate an enhanced feature map.

[0116] In a further optional embodiment, the step of performing multiple non-linear transformations on the mesh surface image feature map to generate a main branch feature map includes:

[0117] Use multiple cascaded bottleneck modules to perform multiple non-linear transformations on the mesh surface image feature map to generate a main branch feature map.

[0118] In this optional embodiment, the mesh surface defect detection model is divided into an input layer for defining the input image size, a backbone network, a neck network, and an anchor-free prediction head. The model of the backbone network is YOLOv8, which includes convolutional layers and cross-stage partial fusion (C2f) modules. In the present invention, the kernel size of the first convolutional layer of the backbone network is changed from 6x6 to 3x3 to reduce the computational amount and improve the flexibility of feature extraction; and all C3 modules in YOLOv8 are replaced with cross-stage partial fusion (C2f) modules. The cross-stage partial fusion module combines the idea of feature extraction and splitting of the cross-stage partial network (CSPNet) and the idea of the residual structure. Its main branch gradient module is the bottleneck module, and the number of stacked modules is controlled by parameters. This design enhances the feature extraction ability of the model, especially when dealing with fine-grained features.

[0119] The processing of the mesh surface image feature map by the cross-stage partial fusion module can be expressed by the formula:

[0120] F main = Bottleneck(F in );

[0121] F aux = Conv1x1(F in );

[0122] F out = Concat(F main , F aux );

[0123] F grid = w gird * (F out + F in );

[0124] Wherein, F in represents the mesh surface image feature map, Bottleneck represents the bottleneck module processing operation, F main represents the main branch feature map, Conv1x1 represents the 1x1 convolution dimensionality reduction operation, F aux represents the auxiliary branch feature map, Concat represents the channel dimension splicing operation, w gird represents the preset weight matrix, and F grid represents the enhanced feature map.

[0125] S103. Extract feature maps of different resolutions from the enhanced feature map, and fuse the feature maps of different resolutions to generate multiple fused feature maps;

[0126] In an alternative embodiment, the steps of extracting feature maps of different resolutions from the enhanced feature map and fusing the feature maps of different resolutions to generate multiple fused feature maps include:

[0127] Extract N feature maps of different resolutions from the enhanced feature map, and sort them in ascending order of resolution to obtain multiple sorted feature maps;

[0128] Traverse the first to the N-1th sorted feature maps:

[0129] Align the size of the nth sorted feature map with the (n+1)th sorted feature map through upsampling operation to obtain an aligned feature map; where n ∈ [1, N-1];

[0130] Perform adaptive scaling on the aligned feature map to obtain a scaled and aligned feature map;

[0131] Add the scaled and aligned feature map and the (n+1)th sorted feature map element by element to generate a fused feature map.

[0132] In this alternative embodiment, step S103 is implemented through a neck network. The neck network introduces a Feature Pyramid Network (FPN) mechanism, and through cross-level feature fusion, it improves the model's detection ability for multi-scale targets, especially small targets.

[0133] It can be seen that in this alternative embodiment, by extracting feature maps of different resolutions, the feature information of the mesh image at different scales can be captured. Sort these feature maps of different resolutions in ascending order of resolution, and then fuse them through upsampling and adaptive scaling methods, the information in the low-resolution feature maps can be gradually transferred to the high-resolution feature maps, realizing cross-scale fusion of features, effectively combining the feature information at different scales, and improving the model's detection and localization ability for mesh defects.

[0134] In yet another alternative embodiment, the steps of performing adaptive scaling on the aligned feature map to obtain a scaled and aligned feature map include:

[0135] Determine the scale adjustment parameter corresponding to the sorted feature map according to the sorted feature map corresponding to the aligned feature map;

[0136] Scale the aligned feature map according to the scale adjustment parameter to obtain a scaled and aligned feature map.

[0137] It can be seen that in this alternative embodiment, by determining the scale adjustment parameter according to the scale information of the sorted feature map, the scale of the aligned feature map can be adaptively adjusted to make it more suitable for fusion with the corresponding sorted feature map. This adaptive scaling method can make feature maps of different scales more coordinated during fusion, improving the effect of feature fusion. At the same time, since the scale adjustment parameter is dynamically determined according to the scale information of the sorted feature map, this scaling method has better flexibility and adaptability.

[0138] In yet another alternative embodiment, the step of determining the scale adjustment parameter corresponding to the sorted feature map according to the aligned feature map corresponding thereto includes:

[0139] Determine the scaling ratio r of the (n + 1)-th sorted feature map to the mesh image feature map n+1 ;

[0140] Determine the scale adjustment parameter s corresponding to the (n + 1)-th sorted feature map according to the following formula n+1 :

[0141] s n+1 = α·(1 - r n+1 ) + β;

[0142] where both α and β are preset hyperparameters.

[0143] In this alternative embodiment, α and β are used to control the degree of scale adjustment, and their values can be set through experiments or experience. α determines the influence degree of the scaling ratio r n+1 on the final scale adjustment parameter s n+1 . A larger α will make s n+1 more dependent on the change of r n+1 , thus generating a larger weight difference between feature maps of different resolutions, which may help to more finely adjust the fusion weights of feature maps of different resolutions. Especially when dealing with multi-scale targets, a smaller α will make s n+1 change more gently, reducing the weight difference between different resolutions, which can avoid some features at certain resolutions from being overly weakened or enhanced. β provides a basic weight to ensure that even at the smallest scaling ratio, s n+1 will not be close to 0. A larger β will make the s n+1 values corresponding to feature maps of all resolutions on the high side, increasing the weight of low-resolution feature maps (such as the original image), which helps to retain more detail information. A smaller β will make the s n+1 values corresponding to feature maps of all resolutions on the low side, reducing the weight of low-resolution feature maps, which may lead to the loss of some detail information.

[0144] It can be seen that the scale adjustment in this alternative embodiment is more adaptable to different mesh images and feature maps, improving the performance of feature fusion. This method of determining the scale adjustment parameters can adaptively adjust the scale of the aligned feature map to make it more matched with the corresponding sorted feature map, thereby improving the effect of feature fusion.

[0145] S104. Predict the state category and defect location of the mesh image feature map based on the fused feature map.

[0146] In this alternative embodiment, the state category represents the state of the drum mesh. In this embodiment, the state categories include normal state, damaged state, blocked state, and damaged and blocked state. The defect location represents the area where loss and / or blockage occur when the drum mesh is in an abnormal state.

[0147] In an alternative embodiment, the step of predicting the state category and defect location of the mesh image feature map based on the fused feature map includes:

[0148] Predict the class probability, center point offset, and bounding box of the fused feature map through an anchor-free prediction head;

[0149] Calculate the defect category of the drum mesh image to be detected according to the class probability, and calculate the defect location and size of the drum mesh image to be detected according to the center point offset and the bounding box.

[0150] It can be seen that in this alternative embodiment, by using an anchor-free prediction head, the category, location, and size information of the defect can be directly predicted from the fused feature map without relying on predefined anchor boxes. This anchor-free prediction method can simplify the model design, reduce the number of hyperparameters, and improve the flexibility and adaptability of the model. At the same time, by jointly predicting the category, location, and size of the defect, the spatial information of the defect can be better captured, improving the detection accuracy.

[0151] In another alternative embodiment, the step of predicting the class probability, center point offset, and bounding box of the fused feature map through an anchor-free prediction head includes:

[0152] Predict the class probability of the fused feature map through an anchor-free prediction head according to the following formula Center point offset And bounding box

[0153]

[0154] where σ represents the Sigmoid activation function, which maps the predicted value to the range [0,1], and W classis the class weight matrix, which is used to enhance the detection ability for specific defect categories, such as corrosion, cracks, blockages, etc. W1 is the class probability weight matrix, W2 is the center point offset weight matrix, W3 is the scale weight matrix, and F grid is the enhanced feature map extracted from the fused feature map.

[0155] It can be seen that in this alternative embodiment, by using different weight matrices to predict class probabilities, center point offsets, and bounding boxes respectively, the model can more flexibly adjust the degree of attention to different tasks. The Sigmoid activation function can map class probabilities to the range of [0, 1], improving the stability of predictions. By extracting the enhanced feature map from the fused feature map as input, the results of feature fusion can be fully utilized to improve the accuracy of predictions.

[0156] In yet another alternative embodiment, the mesh defect detection model is trained according to a predefined loss function; wherein, the loss function is the weighted sum of the distribution focal loss function and the smooth loss function.

[0157] It can be seen that in this alternative embodiment, by combining the distribution focal loss function and the smooth loss function, the model can simultaneously optimize the prediction of defect categories and the regression of defect positions and sizes. The distribution focal loss function can effectively handle the class imbalance problem and improve the attention to difficult-to-classify samples. The smooth loss function can reduce the errors in position and size predictions and improve the detection accuracy. By means of weighted summation, the contributions of the two loss functions can be balanced to obtain better training results.

[0158] In yet another alternative embodiment, the distribution focal loss function L df is:

[0159]

[0160] wherein, I represents the number of mesh image feature maps in the training dataset, J represents the number of status categories, y ij represents the label of the i-th mesh image feature map corresponding to the j-th status category, p ij represents the probability that the i-th mesh image feature map is the j-th status category, γ represents the focusing parameter, and α class represents the balance factor.

[0161] It can be seen that the focusing parameter in this optional embodiment can control the degree of attention of the loss function to difficult-to-classify samples. When the focusing parameter is greater than 1, the model can pay more attention to difficult-to-classify samples and improve the classification ability for these samples. The balance factor can balance the contributions of positive and negative samples to the loss function and alleviate the problem of class imbalance. By reasonably setting the focusing parameter and the balance factor, the distribution focal loss function can be made more adaptable to different data distributions and task requirements, improving the performance of the model.

[0162] In yet another optional embodiment, the smoothing loss function L smooth is as follows:

[0163]

[0164] where x represents the error between the predicted center point offset, the bounding box and the ground truth, and W box represents the bounding box regression weight.

[0165] It can be seen that in this optional embodiment, by using a piecewise function, more sensitive punishment can be imposed on small errors, encouraging the model to generate more accurate prediction results. When the error is large, the linear term can avoid the problem of gradient explosion and improve the stability of training. At the same time, introducing the bounding box regression weight can adjust the proportion of position and size prediction in the total loss function, making the model pay more attention to these tasks. The design of the smoothing loss function can effectively optimize the model's prediction of the defect position and size, improving the detection accuracy.

[0166] Please refer to Figure 2 as shown in Figure 2 which is a schematic structural diagram of a drum mesh surface defect detection system disclosed in an embodiment of the present invention, including:

[0167] An acquisition module 201, configured to acquire an image of the drum mesh surface to be detected, perform input adaptation processing on it, and generate a mesh surface image feature map;

[0168] An enhancement module 202, configured to input the mesh surface image feature map into a mesh surface defect detection model for feature fusion enhancement to obtain an enhanced feature map;

[0169] A fusion module 203, configured to extract feature maps with different resolutions from the enhanced feature map, fuse the feature maps with different resolutions, and generate multiple fused feature maps;

[0170] A prediction module 204, configured to predict the state category and defect position of the mesh surface image feature map according to the fused feature map.

[0171] For the specific limitations of the drum mesh surface defect detection system, reference can be made to the limitations of the drum mesh surface defect detection method in the foregoing text, which will not be elaborated herein. Each module in the above drum mesh surface defect detection system can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above modules can be embedded in the processor of the electronic device in a hardware format or independent thereof, or stored in the memory of the electronic device in a software format, so as to facilitate the processor to call the operations corresponding to each of the above modules.

[0172] It should be noted that, in order to highlight the innovative part of the present invention, modules not closely related to solving the technical problems proposed by the present invention are not introduced in this embodiment, but this does not mean that there are no other modules in this embodiment.

[0173] As Figure 3 shown, the electronic device 1 provided by the present invention may include a memory 11, a processor 12, and a bus, and may further include a computer program stored in the memory 11 and executable on the processor 12, such as a drum mesh surface defect detection program.

[0174] Among them, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, magnetic disk, optical disc, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as the mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the electronic device 1. Further, the memory 11 may include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 can be used not only to store application software installed in the electronic device 1 and various types of data, such as the code for drum mesh surface defect detection, etc., but also to temporarily store data that has been output or will be output.

[0175] In some embodiments, the processor 12 may be composed of an integrated circuit. For example, it may be composed of a single packaged integrated circuit, or may be composed of multiple packaged integrated circuits with the same or different functions, including a combination of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 12 is the control core of the electronic device 1. It uses various interfaces and circuits to connect each component of the entire electronic device 1, and by running or executing programs or modules stored in the memory 11 (such as the drum mesh surface defect detection program, etc.), and calling the data stored in the memory 11, it performs various functions of the electronic device 1 and processes data.

[0176] The processor 12 executes the operating system of the electronic device 1 and various installed application programs. The processor 12 executes the application programs to implement the steps in the above-mentioned drum mesh surface defect detection method.

[0177] Exemplarily, the computer program may be divided into one or more modules. The one or more modules are stored in the memory 11 and executed by the processor 12 to complete this application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and these instruction segments are used to describe the execution process of the computer program in the electronic device 1. For example, the computer program may be divided into an acquisition module 201, an enhancement module 202, a fusion module 203, and a prediction module 204.

[0178] The above-mentioned integrated unit implemented in the form of a software function module can be stored in a computer-readable storage medium. The storage medium may be non-volatile or volatile. The above-mentioned software function module is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a computer device, or a network device, etc.) or a processor to execute some functions of the drum mesh surface defect detection method described in various embodiments of this application.

[0179] In summary, a drum mesh surface defect detection method, system, device, and medium disclosed in the present invention can effectively identify subtle defects, reduce omissions and misjudgments in manual inspections, thereby improving the accuracy of defect detection; the automatic detection method can quickly process a large amount of mesh surface image data, realize real-time or near-real-time defect detection and identification, greatly improving the detection efficiency, and providing a guarantee for timely discovery and repair of defects. Therefore, the present invention effectively overcomes various shortcomings in the prior art and has high industrial utilization value.

[0180] The above embodiments are only illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes made by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A method for detecting defects on the surface of a drum screen, characterized in that, Including: Obtain the image of the drum screen to be detected, perform input adaptation processing on it, and generate a feature map of the screen image; Input the feature map of the screen image into a screen defect detection model for feature fusion enhancement to obtain an enhanced feature map; Extract feature maps with different resolutions from the enhanced feature map, fuse the feature maps with different resolutions, and generate multiple fused feature maps; Predict the state category and defect location of the feature map of the screen image according to the fused feature map.

2. The method for detecting defects on the drum mesh surface according to claim 1, wherein The step of inputting the feature map of the screen image into a screen defect detection model for feature fusion enhancement to obtain an enhanced feature map includes: Input the feature map of the screen image into a screen defect detection model; Perform multiple non-linear transformations on the feature map of the screen image to generate a main branch feature map; Perform dimensionality reduction processing on the feature map of the screen image to generate an auxiliary branch feature map; Perform channel dimension splicing on the main branch feature map and the auxiliary branch feature map, and fuse the splicing result with the feature map of the screen image through a residual addition operation to generate an enhanced feature map.

3. The method for detecting defects on the drum mesh surface according to claim 2, wherein, The step of performing multiple non-linear transformations on the feature map of the screen image to generate a main branch feature map includes: Use multiple cascaded bottleneck modules to perform multiple non-linear transformations on the feature map of the screen image to generate a main branch feature map.

4. A method for detecting defects on the drum mesh surface according to claim 1, characterized in that, The step of extracting feature maps with different resolutions from the enhanced feature map, fusing the feature maps with different resolutions, and generating multiple fused feature maps includes: Extract N feature maps with different resolutions from the enhanced feature map, and sort them in ascending order of resolution to obtain multiple sorted feature maps; Traverse the first to the N-1th sorted feature maps: Align the size of the nth sorted feature map with the (n + 1)th sorted feature map through an upsampling operation to obtain an aligned feature map; where n ∈ [1, N-1]; Perform adaptive scaling on the aligned feature map to obtain a scaled and aligned feature map; Add the scaled and aligned feature map and the (n + 1)th sorted feature map element by element to generate a fused feature map.

5. A method for detecting defects on the drum mesh surface according to claim 4, characterized in that, The step of performing adaptive scaling on the aligned feature map to obtain a scaled and aligned feature map includes: Determine the scale adjustment parameter corresponding to the sorted feature map according to the sorted feature map corresponding to the aligned feature map; Scale the aligned feature map according to the scale adjustment parameter to obtain a scaled and aligned feature map.

6. A method for detecting defects on the drum mesh surface according to claim 4, characterized in that, The step of determining the scale adjustment parameter corresponding to the sorted feature map according to the sorted feature map corresponding to the aligned feature map includes: Determine the scaling ratio r of the (n + 1)-th sorted feature map and the mesh image feature map n+1 ; Determine the scale adjustment parameter s corresponding to the (n + 1)-th sorted feature map according to the following formula n+1 :[[]]END]] s n+1 ＝α·(1-r n+1 )+β； Where both α and β are preset hyperparameters.

7. A method for detecting defects on the drum mesh surface according to claim 1, characterized in that, The step of predicting the state category and defect location of the feature map of the screen image according to the fused feature map includes: Predict the class probability, center point offset, and bounding box of the fused feature map through an anchor-free prediction head; Calculate the defect category of the drum screen image to be detected according to the class probability, and calculate the defect location and size of the drum screen image to be detected according to the center point offset and the bounding box.

8. A method for detecting defects on the drum mesh surface according to claim 7, characterized in that, Predict the class probabilities of the fused feature map through an anchor-free prediction head according to the following formula Center point offset and bounding box : Among them, σ represents the Sigmoid activation function, W class is the class weight matrix, W1 is the class probability weight matrix, W2 is the center point offset weight matrix, W3 is the scale weight matrix, F grid is the enhanced feature map extracted from the fused feature map.

9. A method for detecting defects on the surface of a drum mesh according to claim 1, characterized in that The screen defect detection model is trained according to a predefined loss function; where the loss function is a weighted sum of a distribution focal loss function and a smooth loss function.

10. A method for detecting defects on the surface of a drum mesh according to claim 9, characterized in that, The distribution focal loss function L df is as follows: Among them, I represents the number of mesh image feature maps in the training dataset, J represents the number of state categories, and y ij represents the label of the i-th mesh image feature map corresponding to the j-th state category, and p ij represents the probability that the i-th mesh image feature map is of the j-th state category, γ represents the focusing parameter, and α class represents the balance factor.

11. A method for detecting defects on the drum mesh surface according to claim 9, characterized in that, The smoothing loss function L smooth is as follows: Among them, x represents the offset of the predicted center point, the error between the bounding box and the ground truth, and W box represents the bounding box regression weight.

12. A method for detecting defects on the surface of a drum mesh according to claim 1, characterized in that, The steps of obtaining the drum mesh surface image to be detected, performing input adaptation processing on it, and generating the mesh surface image feature map include: Obtain the drum mesh surface image to be detected; Perform grayscale conversion on the drum mesh surface image to be detected; And / or perform feature scaling on the drum mesh surface image to be detected; Generate the mesh surface image feature map.

13. The method for detecting defects on the drum mesh surface according to claim 1, characterized in that, The steps of obtaining the drum mesh surface image to be detected, performing input adaptation processing on it, and generating the mesh surface image feature map include: Obtain the drum mesh surface image to be detected; Perform histogram equalization processing and / or contrast enhancement processing on the drum mesh surface image to be detected; Generate the mesh surface image feature map.

14. A method for detecting defects on the surface of a drum mesh according to claim 1, characterized in that, The steps of obtaining the drum mesh surface image to be detected, performing input adaptation processing on it, and generating the mesh surface image feature map include: Obtain the drum mesh surface image to be detected; Use an edge detection algorithm to extract edge features from the drum mesh surface image to be detected; Generate the mesh surface image feature map.

15. A method for detecting defects on the surface of a drum mesh according to claim 1, characterized in that, The steps of obtaining the drum mesh surface image to be detected include: Judge whether the size of the input drum mesh surface image exceeds a preset size threshold; When it exceeds, divide the drum mesh surface image into multiple images with sizes smaller than the size threshold to obtain multiple drum mesh surface images to be detected.

16. A drum net surface defect detection system, characterized in that, Include: An acquisition module, configured to obtain the drum mesh surface image to be detected, perform input adaptation processing on it, and generate the mesh surface image feature map; An enhancement module, configured to input the mesh surface image feature map into a mesh surface defect detection model for feature fusion enhancement to obtain an enhanced feature map; A fusion module, configured to extract feature maps with different resolutions from the enhanced feature map, fuse the feature maps with different resolutions, and generate multiple fused feature maps; A prediction module, configured to predict the state category and defect location of the mesh surface image feature map according to the fused feature map.

17. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, the steps of the drum mesh surface defect detection method according to any one of claims 1 to 15 are implemented.

18. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the steps of the drum mesh surface defect detection method according to any one of claims 1 to 15 are implemented.

Images