A defect monitoring method based on a visual large model

Abstract

The application discloses a defect monitoring method based on a visual large model and relates to the technical field of defect detection. An original image is collected and image enhancement is performed to obtain an initial image; the initial image is continuously substituted into three convolution layers to obtain first features, and the first features are substituted into a two-dimensional maximum pooling layer to obtain second features; the second features are sequentially substituted into three light feature extraction modules to obtain first light features, second light features and third light features; the first light features, the second light features and the third light features are substituted into a feature fusion module to obtain target fusion features; and a preset database is determined according to the target fusion features to determine a fault defect. Through image enhancement, the quality of the original image is directly optimized, secondary collection caused by uneven illumination is avoided, and time cost is reduced; then features are extracted from the enhanced image, the efficiency and accuracy of feature extraction are improved, and the extracted features are substituted into a database for comparison, so that the efficiency of defect detection is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of defect detection, and particularly relates to a defect monitoring method based on a visual large model. BACKGROUND

[0002] In many key fields such as modern industrial production, precision manufacturing and quality control, defect monitoring, as a core link to guarantee product quality and improve production efficiency, always occupies a crucial position. In traditional defect detection methods, manual detection, relying on visual observation and experience judgment, has been one of the main detection methods for a long time. However, this method has many drawbacks: its efficiency is extremely low, and in large-scale production scenarios, it is difficult to meet the demand for rapid detection; it is highly subjective, and there are differences in standards and judgments of different detection personnel; and detection personnel are easily affected by physiological fatigue, emotional fluctuations and other factors, resulting in deviation of detection results, making it difficult to guarantee the accuracy and consistency of detection.

[0003] Although the detection method based on traditional computer vision technology overcomes some limitations of manual detection to some extent, it also faces severe challenges. In the face of workpieces with complex structure and texture, the method has great difficulty in feature extraction, and cannot effectively extract feature vectors that accurately reflect the essence of defects, thereby causing obvious upper limit of detection accuracy and making it difficult to meet the requirements of high-precision detection.

[0004] With the vigorous development of deep learning technology, the target detection model based on convolutional neural network has excellent performance in various visual perception tasks due to its powerful feature learning ability. However, most of the workpiece surface defect detection methods based on convolutional neural network on the market can only obtain rough rectangular bounding box information and category information of the defect target, lack accurate description of defect details, and cannot fully exploit and utilize global context information, resulting in low defect detection efficiency. SUMMARY

[0005] The purpose of the present application is to solve the problem of low defect detection efficiency, and a defect monitoring method based on a visual large model is proposed.

[0006] In the implementation of the present application, a defect monitoring method based on a visual large model is proposed, which comprises:

[0007] Collecting an original image, and performing image enhancement on the original image to obtain an initial image;

[0008] Consecutively inputting the initial image into three convolutional layers to obtain first features, and inputting the first features into a two-dimensional maximum pooling layer to obtain second features;

[0009] The second feature is sequentially substituted into three lightweight feature extraction modules to obtain a first lightweight feature, a second lightweight feature and a third lightweight feature;

[0010] The first lightweight feature, the second lightweight feature and the third lightweight feature are substituted into a feature fusion module to obtain a target fusion feature;

[0011] According to the target fusion feature, a preset database is searched, and if a cosine similarity value between a historical target fusion feature in the preset database and the target fusion feature is greater than a preset threshold, it is determined that a fault defect corresponding to the historical target fusion feature is a fault detection result; the preset database stores historical target fusion features corresponding to different steel structures under different fault defects.

[0012] Optionally, the original image is subjected to image enhancement to obtain an initial image, including:

[0013] Through the formula The original image is subjected to brightness correction compensation to obtain a first initial image;

[0014] Wherein, g is the first initial image, is a brightness compensation value of a pixel point i, j in the first initial image, f is the original image, is a brightness value of a pixel point i, j in the original image, and α and β are constant coefficients;

[0015] Through the formula The original image is subjected to logarithmic image conversion to obtain a second initial image;

[0016] Wherein, is a logarithmic conversion value corresponding to a pixel point i, j in the second initial image, and l is the second initial image;

[0017] Through the formula The weight parameters of each pixel point in the first initial image are calculated;

[0018] Wherein, η is a weight adjustment parameter, is a weight corresponding to a pixel point i, j in the first initial image;

[0019] According to the weight parameters of each pixel point, the first initial image and the second initial image are fused to obtain an enhanced image.

[0020] Optionally, the specific working principle of the lightweight feature extraction module specifically includes:

[0021] An input feature is obtained, and after 3x3 convolution, the input feature is substituted into two multi-convolution layers in succession to obtain a first convolution feature;

[0022] The first convolution feature is substituted into a feature weighting layer to obtain a second convolution feature;

[0023] The input feature and the second convolution feature are added to obtain an output feature of the lightweight feature extraction module.

[0024] Optionally, the specific working principle of the multi-convolution layer specifically includes:

[0025] An input feature is obtained, and 1x1 convolution and 3x3 convolution are respectively performed on the input feature to obtain third and fourth convolution features;

[0026] Batch normalization is respectively performed on the input feature, the third convolution feature and the fourth convolution feature, and then feature addition is performed to obtain a first fusion convolution feature;

[0027] ReLU activation is performed on the first fusion convolution feature to obtain an output feature of the multi-convolution layer.

[0028] Optionally, substituting the first convolution feature into the feature weighting layer to obtain the second convolution feature includes:

[0029] X average convolution and Y average convolution are performed on the first convolution feature to obtain X convolution features and Y convolution features;

[0030] The X convolution features and the Y convolution features are respectively substituted into a Sigmoid activation function to obtain X convolution weights and Y convolution weights;

[0031] The first convolution feature is adjusted in weight according to the X convolution weights and the Y convolution weights to obtain a re-weighted feature;

[0032] After average pooling is performed on the re-weighted feature, a Softmax activation function is substituted into to obtain a first attention;

[0033] 3x3 convolution is performed on the first convolution feature to obtain a fifth convolution feature;

[0034] After average pooling is performed on the fifth convolution feature, a Softmax activation function is substituted into to obtain a second attention;

[0035] The re-weighted feature and the second attention are multiplied to obtain a first product feature, and the fifth convolution feature and the first attention are multiplied to obtain a second product feature;

[0036] Element-wise addition is performed on the first product feature and the second product feature, and a Sigmoid activation function is substituted into to obtain a target weight;

[0037] The first convolution feature is adjusted in weight according to the target weight to obtain a second convolution feature.

[0038] Optionally, the first lightweight feature, the second lightweight feature and the third lightweight feature are substituted into a feature fusion module to obtain a target fusion feature, which includes:

[0039] The first lightweight feature is down-sampled to obtain a first down-sampled fusion feature, and the third lightweight feature is up-sampled to obtain a first up-sampled fusion feature;

[0040] The first down-sampled fusion feature, the second lightweight feature and the first up-sampled fusion feature are substituted into a feature enhancement module to obtain a first enhanced feature;

[0041] The first enhanced feature is respectively up-sampled and down-sampled to obtain a second up-sampled feature and a second down-sampled feature;

[0042] The first lightweight feature and the second up-sampled feature are fused to obtain a first fused sampling feature, and the third lightweight feature and the second down-sampled feature are fused to obtain a second fused sampling feature;

[0043] The first fused sampling feature and the second fused sampling feature are respectively substituted into a re-parameterization convolution layer to obtain a first re-parameterization feature and a second re-parameterization feature;

[0044] The first re-parameterization feature is down-sampled to obtain a third down-sampled fusion feature, and the second re-parameterization feature is up-sampled to obtain a third up-sampled fusion feature;

[0045] The third down-sampled fusion feature, the third up-sampled fusion feature and the first enhanced feature are substituted into a feature enhancement module to obtain a target fusion feature.

[0046] Optionally, the input of the feature enhancement module includes a low-scale feature, a middle-scale feature and a high-scale feature; when the high-scale feature is the first down-sampled fusion feature, the middle-scale feature is the second lightweight feature, and the low-scale feature is the first up-sampled fusion feature; when the high-scale feature is the third down-sampled fusion feature, the middle-scale feature is the third up-sampled fusion feature, and the low-scale feature is the first enhanced feature; the working principle of the feature enhancement module specifically includes:

[0047] The low-scale feature, the middle-scale feature and the high-scale feature are respectively segmented by a preset size to obtain a sub-low-scale feature set, a sub-middle-scale feature set and a sub-high-scale feature set;

[0048] obtaining a target sub-low scale feature, a target sub-medium scale feature and a target sub-high scale feature; the target sub-low scale feature is any one in the sub-low scale feature set; the target sub-medium scale feature is a sub-medium scale feature corresponding to the same position of the target sub-low scale feature; and the target sub-high scale feature is a sub-high scale feature corresponding to the same position of the target sub-low scale feature;

[0049] obtaining a target sub-low scale feature, a target sub-medium scale feature and a target sub-high scale feature; the target sub-low scale feature is any one in the sub-low scale feature set; the target sub-medium scale feature is a sub-medium scale feature corresponding to the same position of the target sub-low scale feature; and the target sub-high scale feature is a sub-high scale feature corresponding to the same position of the target sub-low scale feature;

[0050] obtaining a target sub-low scale feature, a target sub-medium scale feature and a target sub-high scale feature; the target sub-low scale feature is any one in the sub-low scale feature set; the target sub-medium scale feature is a sub-medium scale feature corresponding to the same position of the target sub-low scale feature; and the target sub-high scale feature is a sub-high scale feature corresponding to the same position of the target sub-low scale feature;

[0051] obtaining a target sub-low scale feature, a target sub-medium scale feature and a target sub-high scale feature; the target sub-low scale feature is any one in the sub-low scale feature set; the target sub-medium scale feature is a sub-medium scale feature corresponding to the same position of the target sub-low scale feature; and the target sub-high scale feature is a sub-high scale feature corresponding to the same position of the target sub-low scale feature;

[0052] obtaining a target sub-low scale feature, a target sub-medium scale feature and a target sub-high scale feature; the target sub-low scale feature is any one in the sub-low scale feature set; the target sub-medium scale feature is a sub-medium scale feature corresponding to the same position of the target sub-low scale feature; and the target sub-high scale feature is a sub-high scale feature corresponding to the same position of the target sub-low scale feature;

[0053] obtaining a target sub-low scale feature, a target sub-medium scale feature and a target sub-high scale feature; the target sub-low scale feature is any one in the sub-low scale feature set; the target sub-medium scale feature is a sub-medium scale feature corresponding to the same position of the target sub-low scale feature; and the target sub-high scale feature is a sub-high scale feature corresponding to the same position of the target sub-low scale feature;

[0054] Advantages of the present application:

[0055] The present application provides a defect monitoring method based on a visual large model, collects an original image, performs image enhancement on the original image to obtain an initial image, continuously substitutes the initial image into three convolution layers to obtain a first feature, substitutes the first feature into a two-dimensional maximum pooling layer to obtain a second feature, sequentially substitutes the second feature into three lightweight feature extraction modules to obtain a first lightweight feature, a second lightweight feature and a third lightweight feature, substitutes the first lightweight feature, the second lightweight feature and the third lightweight feature into a feature fusion module to obtain a target fusion feature, searches a preset database according to the target fusion feature, if a cosine similarity value between a historical target fusion feature in the preset database and the target fusion feature is greater than a preset threshold value, determines a fault defect corresponding to the historical target fusion feature as a fault detection result, and the preset database stores historical target fusion features corresponding to different steel structures under different fault defects. Through image enhancement, the quality of the original image is directly optimized, secondary collection caused by uneven illumination is avoided, and time cost is reduced. Through feature extraction on the enhanced image, the efficiency and accuracy of feature extraction are improved, and the efficiency of defect detection is improved through comparison of the extracted features with the database. BRIEF DESCRIPTION OF DRAWINGS

[0056] The present application will be further described below with reference to the drawings.

[0057] Figure 1 A flowchart of a defect monitoring method based on a visual large model is provided for an embodiment of the present application.

[0058] Figure 2 A flowchart of target fusion feature acquisition is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application.

[0060] Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0061] An embodiment of the present application provides a defect monitoring method based on a visual large model. Referring to Figure 1 , Figure 1 A flowchart of a defect monitoring method based on a visual large model is provided for an embodiment of the present application. The method comprises the following steps:

[0062] S101, collecting an original image and performing image enhancement on the original image to obtain an initial image;

[0063] S102, continuously substituting the initial image into three convolution layers to obtain first features, and substituting the first features into a two-dimensional maximum pooling layer to obtain second features;

[0064] S103, substituting the second features into three lightweight feature extraction modules in sequence to obtain first lightweight features, second lightweight features, and third lightweight features;

[0065] S104, substituting the first lightweight features, the second lightweight features, and the third lightweight features into a feature fusion module to obtain target fusion features;

[0066] S105, searching a preset database according to the target fusion features, and if a cosine similarity value between a historical target fusion feature in the preset database and the target fusion features is greater than a preset threshold, determining a fault defect corresponding to the historical target fusion feature as a fault detection result;

[0067] The preset database stores historical target fusion features corresponding to different steel structures under different fault defects.

[0068] Based on the defect monitoring method provided by the embodiment of the application, the original image quality is directly optimized through image enhancement, secondary collection caused by uneven illumination is avoided, and time cost is reduced; then, the efficiency and accuracy of feature extraction are improved through feature extraction on the enhanced image, and the efficiency of defect detection is improved through comparison of the extracted features with the database.

[0069] In an implementation manner, the collected original image is an image containing a building workpiece, the building workpiece is a steel structure including a steel beam, a steel column, etc., the fault defect includes a crack, deformation, damage, corrosion and other defects, and the preset threshold is determined by a technician; if the cosine similarity value between each historical target fusion feature in the preset database and the target fusion feature is not greater than the preset threshold, the building workpiece in the currently collected original image does not have a fault defect.

[0070] In an implementation manner, the brightness of the original image can be automatically optimized through image enhancement to reduce false detection caused by uneven illumination and noise interference, and the dependence of subsequent algorithms on image quality is reduced.

[0071] In an implementation manner, the three convolutional layers are respectively a 1*1 convolution, a 3*3 convolution and a 1*1 convolution, and the step length is 2; the basic features are gradually integrated through convolution and pooling operations on the initial image, and the basis for subsequent feature extraction is provided.

[0072] In an implementation manner, the features output by different lightweight modules are fused to combine the bottom details and high-level semantics, avoid one-sidedness of a single feature, generate target fusion features containing multi-dimensional information through channel splicing, weighted fusion and other operations, and make subsequent defect classification or positioning more accurate; the fusion of the features is averaging of the features.

[0073] In an embodiment, the image enhancement on the original image to obtain the initial image includes:

[0074] The brightness correction compensation on the original image to obtain the first initial image is performed through a formula

[0075] Wherein, g is the first initial image, is a brightness compensation value of a pixel point i, j in the first initial image, f is the original image, is a brightness value of a pixel point i, j in the original image, and alpha and beta are constant coefficients.

[0076] The logarithmic image conversion on the original image to obtain the second initial image is performed through a formula

[0077] Wherein, is a logarithmic conversion value corresponding to a pixel point i, j in the second initial image, and l is the second initial image.​​

[0078] The weight parameter of each pixel point in the first initial image is calculated by the formula

[0079] η is a weight adjustment parameter, is the weight corresponding to the pixel point i,j in the first initial image;

[0080] The first initial image and the second initial image are fused according to the weight parameter of each pixel point to obtain an enhanced image.

[0081] In an implementation, α and β are constant coefficients, respectively controlling the contrast and brightness of the image, and the specific values are determined by the technician; the first initial image and the second initial image are fused according to the weight parameter of each pixel point to obtain an enhanced image, and the pixel point fusion formula is , wherein is the brightness compensation value of the pixel point i,j in the first initial image, is the weight corresponding to the pixel point i,j in the first initial image, is the logarithmic transformation value corresponding to the pixel point i,j in the second initial image, is the pixel value corresponding to the pixel point i,j in the enhanced image; and the enhanced image is obtained according to the fusion results of all pixel points.

[0082] In an implementation, the original image is brightness corrected by the formula Nonlinear adjustment can be performed on different brightness regions, the brightness difference can be enlarged for dark regions to make the details clearer, and the overexposure can be avoided for bright regions to retain highlight details; then the original brightness is mapped to the logarithmic space by the formula, so that the numerical interval of dark pixels is enlarged and the interval of bright pixels is compressed.

[0083] In an embodiment, the specific working principle of the lightweight feature extraction module specifically includes:

[0084] The input feature is obtained, and after 3*3 convolution is performed on the input feature, the first convolution feature is obtained by continuously substituting two multi-convolution layers;

[0085] The first convolution feature is substituted into the feature weighting layer to obtain the second convolution feature;

[0086] The input feature and the second convolution feature are added to obtain the output feature of the lightweight feature extraction module.

[0087] In an implementation, the two consecutive multi-convolution layers can abstract the features layer by layer, the first layer extracts basic edges and direction information, and the second layer integrates the basic features to form more complex local patterns; compared with a single deep convolution layer, the cascade of two lightweight convolution layers can realize equivalent feature extraction capability with less parameters, reducing the algorithm power consumption.​

[0088] In an implementation, the feature map of each channel is globally averaged and pooled to obtain channel-level semantic information, and a channel weight is generated through a fully connected layer or a simple convolution to suppress irrelevant channels and enhance target-related channels.

[0089] In an implementation, the input feature is directly added to the second convolution feature to provide a direct gradient path for the deep network, avoid the loss of low-frequency information in convolution processing, and enable the output feature to contain both the global structure of the original input and the local details extracted by convolution.

[0090] In one embodiment, the specific working principle of the multi-convolution layer specifically includes:

[0091] An input feature is obtained, and 1x1 convolution and 3x3 convolution are respectively performed on the input feature to obtain third convolution features and fourth convolution features;

[0092] Batch normalization is performed on the input feature, the third convolution feature, and the fourth convolution feature, respectively, and then the features are added to obtain first fused convolution features;

[0093] ReLU activation is performed on the first fused convolution features to obtain output features of the multi-convolution layer.

[0094] In an implementation, batch normalization is performed on the input feature, the third convolution feature, and the fourth convolution feature, respectively, to keep the mean and variance of the input features of each layer stable and avoid gradient disappearance caused by increasing network depth.

[0095] In an implementation, the normalized features make the input of the activation function closer to the effective interval, improving the training efficiency; the original input feature, the 1x1 convolution feature, and the 3x3 convolution feature are added to avoid the loss of low-frequency information caused by convolution operation; and the added features are activated by ReLU to introduce nonlinearity, which can map the linearly combined features to a more complex space and improve the expression ability for non-linear patterns such as defects and targets.

[0096] In one embodiment, substituting the first convolution feature into the feature weighting layer to obtain the second convolution feature includes:

[0097] X average convolution and Y average convolution are performed on the first convolution feature to obtain X convolution features and Y convolution features;

[0098] The X convolution features and the Y convolution features are respectively substituted into Sigmoid activation functions to obtain X convolution weights and Y convolution weights;

[0099] The first convolution feature is adjusted in weight according to the X convolution weights and the Y convolution weights to obtain a re-weighted feature;

[0100] The first attention is obtained by performing average pooling on the reweighted feature and inputting the reweighted feature into a Softmax activation function;

[0101] The fifth convolutional feature is obtained by performing 3*3 convolution on the first convolutional feature;

[0102] The second attention is obtained by performing average pooling on the fifth convolutional feature and inputting the fifth convolutional feature into a Softmax activation function;

[0103] The first product feature is obtained by multiplying the reweighted feature and the second attention, and the second product feature is obtained by multiplying the fifth convolutional feature and the first attention;

[0104] The target weight is obtained by performing element-wise addition on the first product feature and the second product feature and inputting the first product feature and the second product feature into a Sigmoid activation function;

[0105] The second convolutional feature is obtained by performing weight adjustment on the first convolutional feature according to the target weight.

[0106] In an implementation, the X average convolution and the Y average convolution on the first convolutional feature are performed to aggregate features along a horizontal direction and capture horizontal texture, and to aggregate features along a vertical direction and capture vertical structure; the X / Y convolutional weight is generated by a Sigmoid activation, and the reweighted feature is obtained by weighting the importance of each region in the first convolutional feature.

[0107] In an implementation, the first attention is generated by performing average pooling on the reweighted feature and inputting the reweighted feature into a Softmax, to identify the overall category to which the defect belongs and suppress irrelevant region interference; the second attention is generated by performing average pooling on the fifth convolutional feature and inputting the fifth convolutional feature into a Softmax, to focus on local details and locate the accurate position of the defect, and to form a complement of global semantics and local positioning with the first attention.

[0108] In an implementation, the target weight is dynamically allocated according to feature importance, a low weight is given to a noise region, and a high weight is given to a defect feature, to reduce the false detection rate; the target weight is generated by inputting the sum of the two product features into a Sigmoid, to simultaneously integrate global semantic constraint and local detail positioning, and to avoid one-sidedness of a single attention; the second convolutional feature after weight adjustment simultaneously contains defect position information reinforced by spatial weight and defect category information screened by semantic attention.

[0109] In one embodiment, referring to Figure 2 , Figure 2 A flowchart for obtaining a target fusion feature is provided, including:

[0110] S1041, down-sampling the first lightweight feature to obtain a first down-sampled fusion feature, and up-sampling the third lightweight feature to obtain a first up-sampled fusion feature;

[0111] S1042, the first down-sampling fusion feature, the second lightweight feature and the first up-sampling fusion feature are substituted into the feature enhancement module to obtain a first enhanced feature;

[0112] S1043, the first enhanced feature is respectively up-sampled and down-sampled to obtain a second up-sampling feature and a second down-sampling feature;

[0113] S1044, the first lightweight feature and the second up-sampling feature are fused to obtain a first fusion sampling feature, and the third lightweight feature and the second down-sampling feature are fused to obtain a second fusion sampling feature;

[0114] S1045, the first fusion sampling feature and the second fusion sampling feature are respectively substituted into a re-parameterization convolution layer to obtain a first re-parameterization feature and a second re-parameterization feature;

[0115] S1046, the first re-parameterization feature is down-sampled to obtain a third down-sampling fusion feature, and the second re-parameterization feature is up-sampled to obtain a third up-sampling fusion feature;

[0116] S1047, the third down-sampling fusion feature, the third up-sampling fusion feature and the first enhanced feature are substituted into the feature enhancement module to obtain a target fusion feature.

[0117] In an implementation manner, the re-parameterization convolution layer is a RepC3 module; the first down-sampling fusion feature, the second lightweight feature and the first up-sampling fusion feature are fused through the feature enhancement module, so as to realize semantic understanding of low-scale features guided by high-scale semantics, positioning accuracy of high-scale features enhanced by low-scale details, and the output first enhanced feature containing global semantics and local details.

[0118] In an implementation manner, the enhanced feature participates in fusion again after being up-sampled and down-sampled, forming a closed-loop operation of feature enhancement, sampling and re-enhancement, and multi-level recognition of complex defects is extracted.

[0119] In an embodiment, the input of the feature enhancement module includes low-scale features, medium-scale features and high-scale features; when the high-scale feature is the first down-sampling fusion feature, the medium-scale feature is the second lightweight feature, and the low-scale feature is the first up-sampling fusion feature; when the high-scale feature is the third down-sampling fusion feature, the medium-scale feature is the third up-sampling fusion feature, and the low-scale feature is the first enhanced feature; the working principle of the feature enhancement module specifically includes:

[0120] The low-scale features, the medium-scale features and the high-scale features are respectively segmented by a preset size to obtain a set of sub-low-scale features, a set of sub-medium-scale features and a set of sub-high-scale features;

[0121] The target sub-low scale feature, the target sub-medium scale feature, and the target sub-high scale feature are obtained; the target sub-low scale feature is any one in the sub-low scale feature set; the target sub-medium scale feature is a sub-medium scale feature corresponding to the same position as the target sub-low scale feature; and the target sub-high scale feature is a sub-high scale feature corresponding to the same position as the target sub-low scale feature;

[0122] The target medium scale weight is obtained by substituting the target sub-medium scale feature into a Sigmoid activation function;

[0123] The first scale fusion feature is obtained by fusing the target sub-low scale feature and the target sub-medium scale feature according to the target medium scale weight;

[0124] The second scale fusion feature is obtained by fusing the target sub-high scale feature and the target sub-medium scale feature according to the target medium scale weight;

[0125] The target sub-scale fusion feature is obtained by adding the first scale fusion feature and the second scale fusion feature;

[0126] The output feature of the feature enhancement module is obtained by splicing all the target sub-scale fusion features.

[0127] In an implementation manner, the features corresponding to the low, medium, and high scales are divided into sub-feature sets according to a preset size, so that the sub-features of different scales strictly correspond in spatial positions, and the positional deviation problem caused by scale difference in traditional feature fusion is avoided; the preset size is determined by a technician.

[0128] In an implementation manner, the first scale fusion feature is equal to the target medium scale weight multiplied by the low scale feature plus the correction weight multiplied by the medium scale feature; the correction weight is one minus the target medium scale weight; through such fusion, the details of the low scale feature are selectively enhanced by the medium scale weight, while the semantic constraint of the medium scale is retained, and the interference of detail noise is avoided; similarly, the second scale fusion feature is equal to the target medium scale weight multiplied by the high scale feature plus the correction weight multiplied by the medium scale feature; the correction weight is one minus the target medium scale weight; the global semantics of the high scale are transmitted to the medium scale feature through the weight, so as to guide the semantic understanding of local details.

[0129] In an implementation manner, after the two fusion features are added, the combination of the low scale details and the medium scale semantics is contained, and the guidance of the high scale semantics to the medium scale is also contained; each sub-block independently performs cross-scale fusion, and is spliced into a complete feature map, so that the feature fusion in a local region is optimized, the memory occupation and the single calculation amount are reduced, and the operation speed is improved.

[0130] The above has been described in detail one embodiment of the present application, but the content is only the preferred embodiment of the present application, cannot be considered for limiting the scope of the present application. Any equivalent changes and improvements made in the scope of the present application, should still belong to the scope of the present application.

Claims

1. A defect monitoring method based on a large visual model, characterized in that, The method includes: Acquire the original image, and perform image enhancement on the original image to obtain the initial image; The initial image is successively fed into three convolutional layers to obtain the first feature, and the first feature is then fed into a two-dimensional max pooling layer to obtain the second feature. The second feature is substituted into the three lightweight feature extraction modules in sequence to obtain the first lightweight feature, the second lightweight feature, and the third lightweight feature; The first lightweight feature, the second lightweight feature, and the third lightweight feature are substituted into the feature fusion module to obtain the target fused feature; The target fusion feature is searched in a preset database. If the cosine similarity value between the target fusion feature and the historical target fusion feature in the preset database is greater than a preset threshold, then the fault defect corresponding to the historical target fusion feature is determined as the fault detection result. The preset database stores historical target fusion features corresponding to different steel structures under different fault defects. The specific working principle of the lightweight feature extraction module includes: The input features are obtained, and after performing a 3×3 convolution on the input features, they are continuously substituted into two multi-convolutional layers to obtain the first convolutional features. Substitute the first convolutional feature into the feature weighting layer to obtain the second convolutional feature; The input features and the second convolutional features are added together to obtain the output features of the lightweight feature extraction module; The specific working principle of the multi-convolutional layer includes: Obtain the input features, and perform 1×1 convolution and 3×3 convolution on the input features respectively to obtain the third convolution feature and the fourth convolution feature; After batch normalization of the input features, the third convolutional features, and the fourth convolutional features, the features are summed to obtain the first fused convolutional feature; The output features of the multi-convolutional layer are obtained by performing ReLU activation on the first fused convolutional features.

2. The defect monitoring method based on a large visual model according to claim 1, characterized in that, Image enhancement of the original image to obtain the initial image includes: Through formula The original image is subjected to brightness correction compensation to obtain a first initial image; Where g is the first initial image, Let f be the brightness compensation value for pixels i and j in the first initial image, and let f be the original image. Let α and β be the brightness values ​​of pixels i and j in the original image, and α and β be constant coefficients. Through formula The original image is logarithmically transformed to obtain a second initial image; in, is the logarithmic transformation value corresponding to pixel points i and j in the second initial image, and l is the second initial image; Through formula Calculate the weight parameters for each pixel in the first initial image; Where η is the weight adjustment parameter, Let i be the weight of pixel i and j in the first initial image; The enhanced image is obtained by fusing the first initial image and the second initial image based on the weight parameters of each pixel.

3. The defect monitoring method based on a large visual model according to claim 1, characterized in that, Substituting the first convolutional features into the feature weighting layer yields the second convolutional features, including: Perform X-mean convolution and Y-mean convolution on the first convolutional feature to obtain X-convolutional feature and Y-convolutional feature; Substitute the X convolutional features and the Y convolutional features into the Sigmoid activation function to obtain the X convolutional weights and the Y convolutional weights, respectively. The first convolutional feature is weighted according to the X convolutional weight and the Y convolutional weight to obtain the reweighted feature; After performing average pooling on the reweighted features, the first attention is obtained by inputting the softmax activation function; The fifth convolutional feature is obtained by performing a 3×3 convolution on the first convolutional feature. After performing average pooling on the fifth convolutional feature, the second attention is obtained by inputting the softmax activation function; The first product feature is obtained by multiplying the reweighted feature and the second attention; the second product feature is obtained by multiplying the fifth convolution feature and the first attention. The target weights are obtained by performing element-wise addition on the first product features and the second product features, followed by the Sigmoid activation function; The first convolutional feature is weighted according to the target weight to obtain the second convolutional feature.

4. The defect monitoring method based on a large visual model according to claim 1, characterized in that, Substituting the first lightweight feature, the second lightweight feature, and the third lightweight feature into the feature fusion module yields the target fused features, including: The first lightweight feature is downsampled to obtain a first downsampled fusion feature, and the third lightweight feature is upsampled to obtain a first upsampled fusion feature; Substituting the first downsampling fusion feature, the second lightweight feature, and the first upsampling fusion feature into the feature enhancement module yields the first enhanced feature; The first enhanced feature is upsampled and downsampled to obtain the second upsampled feature and the second downsampled feature, respectively; The first lightweight feature and the second upsampled feature are fused to obtain a first fused sampling feature, and the third lightweight feature and the second downsampled feature are fused to obtain a second fused sampling feature; The first fused sampling feature and the second fused sampling feature are respectively substituted into the reparameterized convolutional layer to obtain the first reparameterized feature and the second reparameterized feature; The first reparameterized feature is downsampled to obtain the third downsampled fusion feature, and the second reparameterized feature is upsampled to obtain the third upsampled fusion feature; The target fusion feature is obtained by substituting the third downsampling fusion feature, the third upsampling fusion feature, and the first enhancement feature into the feature enhancement module.

5. The defect monitoring method based on a large visual model according to claim 4, characterized in that, The inputs to the feature enhancement module include low-scale features, medium-scale features, and high-scale features; When the high-scale feature is the first downsampled fusion feature, the mid-scale feature is the second lightweight feature, and the low-scale feature is the first upsampled fusion feature; When the high-scale feature is the third downsampled fusion feature, the mid-scale feature is the third upsampled fusion feature, and the low-scale feature is the first enhanced feature; The working principle of the feature enhancement module specifically includes: The low-scale features, medium-scale features, and high-scale features are segmented by preset size pairs to obtain sub-low-scale feature sets, sub-medium-scale feature sets, and sub-high-scale feature sets, respectively. Obtain target sub-low-scale features, target sub-medium-scale features, and target sub-high-scale features; the target sub-low-scale features are any one of the sub-low-scale features in the set; the target sub-medium-scale features are the sub-medium-scale features corresponding to the same position as the target sub-low-scale features; the target sub-high-scale features are the sub-high-scale features corresponding to the same position as the target sub-low-scale features. Substitute the target sub-mesoscale features into the Sigmoid activation function to obtain the target mesoscale weights; The first scale fusion feature is obtained by fusing the target sub-low-scale feature and the target sub-medium-scale feature according to the target mesoscale weight; The second scale fusion feature is obtained by fusing the target sub-high-scale feature and the target sub-medium-scale feature according to the target mesoscale weight; The target sub-scale fusion feature is obtained by adding the first scale fusion feature and the second scale fusion feature; The output features of the feature enhancement module are obtained by stitching together the fused features of all target sub-scales.

Images