Surface defect detection method and device based on multi-scale adaptive guidance and medium

Abstract

The application discloses a surface defect detection method and device based on multi-scale adaptive guidance and a medium, relates to the technical field of surface defect detection, and comprises the following steps: acquiring an image of a to-be-detected object and inputting the image into a pre-trained surface defect detection network to predict a surface defect detection result of the to-be-detected object; wherein a multi-scale adaptive guidance subnetwork is introduced between a multi-scale feature extraction subnetwork and a feature fusion subnetwork in the surface defect detection network to reduce the loss of weak and small target information; a full-dimensional dynamic feature extraction module is introduced in the feature fusion subnetwork to enhance the feature extraction capability for multi-scale targets and complex shape targets; and finally, a modulation factor capable of changing weight parameters is introduced in a boundary box regression loss term, so that when the surface defect detection network is used for defect detection, the accuracy of surface defect detection is improved, and the missed detection rate and the false detection rate of industrial surface defect detection are reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of surface defect detection, in particular to a surface defect detection method based on multi-scale adaptive guidance, a device and a medium. BACKGROUND

[0002] Surface defect detection is a technology for identifying and locating surface flaws of products, which is widely used in industrial production and is of great significance for ensuring product quality, reducing production cost and improving production efficiency. However, the traditional manual inspection method has many shortcomings, not only low efficiency, but also easily affected by subjective factors, resulting in unstable detection results. With the continuous development of industrial automation and intelligentization, the industrial field urgently needs an efficient, stable and automated technology to replace the traditional manual detection method.

[0003] In recent years, the industrial surface defect detection method based on deep learning has been widely used in modern manufacturing industry due to its high precision, high robustness and automation capability. However, the traditional deep learning technology still has the problems of high missing rate and false detection rate when facing weak texture, multi-scale and complex shape target defects in actual industrial scene application. SUMMARY

[0004] The purpose of the present application is to provide a surface defect detection method based on multi-scale adaptive guidance, a device and a medium, which can improve the precision of surface defect detection and reduce the missing rate and false detection rate of industrial surface defect detection.

[0005] To achieve the above purpose, the present application provides the following solutions:

[0006] In a first aspect, the present application provides a surface defect detection method based on multi-scale adaptive guidance, comprising:

[0007] obtaining an image of a to-be-detected object;

[0008] inputting the image of the to-be-detected object into a pre-trained surface defect detection network to predict a surface defect detection result of the to-be-detected object; the surface defect detection network comprises a multi-scale feature extraction subnetwork, a multi-scale adaptive guidance subnetwork, a feature fusion subnetwork and a detection head subnetwork connected in turn; the multi-scale adaptive guidance subnetwork is used for adaptively fusing a plurality of feature maps of different scales extracted by the multi-scale feature extraction subnetwork to obtain adaptive fusion feature maps of different scales.

[0009] In a second aspect, the present application provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned surface defect detection method based on multi-scale adaptive guidance.

[0010] In a third aspect, the present application provides a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the surface defect detection method based on multi-scale adaptive guidance described above.

[0011] According to the specific embodiments provided in the present application, the following technical effects are disclosed:

[0012] The present application provides a surface defect detection method, device and medium based on multi-scale adaptive guidance, an image of a to-be-detected object is obtained and input into a pre-trained surface defect detection network to predict a surface defect detection result of the to-be-detected object; wherein a multi-scale adaptive guidance subnetwork is introduced between a multi-scale feature extraction subnetwork and a feature fusion subnetwork in the surface defect detection network, which effectively integrates multi-scale feature information, reduces the loss of weak texture defects and tiny defects, improves the accuracy of surface defect detection, and reduces the missed detection rate and false detection rate of industrial surface defect detection. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0014] Figure 1 An application environment diagram of a surface defect detection method based on multi-scale adaptive guidance according to an embodiment of the present application;

[0015] Figure 2 A flowchart of a surface defect detection method based on multi-scale adaptive guidance according to an embodiment of the present application;

[0016] Figure 3 A to-be-detected object image diagram according to an embodiment of the present application;

[0017] Figure 4 A surface defect detection result diagram of a to-be-detected object according to an embodiment of the present application;

[0018] Figure 5 A structure diagram of a surface defect detection network according to an embodiment of the present application;

[0019] Figure 6 A structure diagram of a multi-scale feature extraction subnetwork according to an embodiment of the present application;

[0020] Figure 7A structural diagram of a multi-scale adaptive guiding sub-network provided by an embodiment of the present application is shown in FIG. 1.

[0021] Figure 8 A structural diagram of a multi-scale adaptive fusion module provided by an embodiment of the present application is shown in FIG. 2.

[0022] Figure 9 A structural diagram of a full-dimensional dynamic feature extraction module provided by an embodiment of the present application is shown in FIG. 3.

[0023] Figure 10 A full-dimensional dynamic convolution structural diagram provided by an embodiment of the present application is shown in FIG. 4.

[0024] Figure 11 A structural diagram of a computer device provided by an embodiment of the present application is shown in FIG. 5. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0026] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0027] The surface defect detection method based on multi-scale adaptive guiding provided by the embodiments of the present application can be applied to, for example, Figure 1The application environment shown is illustrated. The terminal communicates with the server via a network. A data storage system stores the data the server needs to process. This data storage system can be set up independently, integrated into the server, or located in the cloud or on another server. The terminal can send an image of the object to be detected to the server. After receiving the image, the server inputs it into a pre-trained surface defect detection network to predict the surface defect detection result. The surface defect detection network includes a multi-scale feature extraction sub-network, a multi-scale adaptive guidance sub-network, a feature fusion sub-network, and a detection head sub-network connected in sequence. The multi-scale adaptive guidance sub-network adaptively fuses multiple feature maps of different scales extracted by the multi-scale feature extraction sub-network to obtain adaptive fused feature maps of different scales. The server can then feed back the obtained surface defect detection result to the terminal. Furthermore, in some embodiments, the multi-scale adaptive guidance-based surface defect detection method can also be implemented independently by the server or the terminal. For example, the terminal can directly perform multi-scale adaptive guidance-based surface defect detection on the image of the object to be detected, or the server can obtain the image of the object to be detected from the data storage system and perform multi-scale adaptive guidance-based surface defect detection.

[0028] The terminal can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. The server can be a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0029] In one exemplary embodiment, such as Figure 2 As shown, a surface defect detection method based on multi-scale adaptive guidance is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 The following steps, 101 to 102, are used as an example to illustrate the process of using a server in the example.

[0030] Step 101: Obtain the image of the object to be detected. For example... Figure 3 The image shown is from a dataset of surface defects in industrial steel.

[0031] Step 102: Input the image of the object to be detected into the pre-trained surface defect detection network to predict the surface defect detection result of the object, such as... Figure 4 As shown. Figure 5As shown in the figure, the surface defect detection network comprises, in sequence, a multi-scale feature extraction subnetwork, a multi-scale adaptive guidance subnetwork, a feature fusion subnetwork, and a detection head subnetwork; the multi-scale adaptive guidance subnetwork is configured to adaptively fuse a plurality of feature maps of different scales extracted by the multi-scale feature extraction subnetwork, to obtain adaptive fusion feature maps of different scales.

[0032] The steps 101 to 102 are implemented to obtain the image of the object to be detected and input the image into the pre-trained surface defect detection network, to predict the surface defect detection result of the object to be detected; wherein, the multi-scale adaptive guidance subnetwork is introduced between the multi-scale feature extraction subnetwork and the feature fusion subnetwork in the surface defect detection network, which effectively integrates multi-scale feature information, reduces the loss of weak texture defects and tiny defects, improves the accuracy of surface defect detection, and reduces the missed detection rate and false detection rate of industrial surface defect detection.

[0033] In another exemplary embodiment of the present application, in step 102, the image of the object to be detected is input into the pre-trained surface defect detection network to predict the surface defect detection result of the object to be detected, which specifically comprises:

[0034] (1) inputting the image of the object to be detected into the multi-scale feature extraction subnetwork to extract a plurality of feature maps of different scales.

[0035] (2) adaptively fusing the plurality of feature maps of different scales by using the multi-scale adaptive guidance subnetwork to obtain adaptive fusion feature maps of different scales.

[0036] (3) splicing and fusing the adaptive fusion features of different scales by using the feature fusion subnetwork to obtain splicing fusion feature maps.

[0037] (4) inputting the splicing fusion feature maps into the detection head subnetwork to output the surface defect detection result of the object to be detected.

[0038] In another exemplary embodiment of the present application, as shown in the figure, Figure 6 The multi-scale feature extraction subnetwork comprises N FasterNet modules connected in series (corresponding to FasterNet Block in Figure 6 The input of the first FasterNet module is the image of the object to be detected; each FasterNet module outputs a feature map of a different scale. As an example, N is 4, Figure 6 four feature maps of different scales are shown in the figure, which correspond to C1 to C4 in Figure 6 Figure 6 ​In the specific implementation, the input end of the first FasterNet module is further connected with an Embedding module, and a Merging module is further arranged between two adjacent FasterNet modules. Figure 6 H and W in the formula respectively represent the length and width of an image.

[0039] In another exemplary embodiment of the present application, as shown in Figure 7 the multi-scale adaptive guiding sub-network comprises N-2 multi-scale adaptive fusion modules. As an example, N is 4, Figure 7 two multi-scale adaptive fusion modules (corresponding to MAG in Figure 7 ) are shown in the formula.

[0040] As shown in Figure 8 , each multi-scale adaptive fusion module comprises an up-sampling layer (corresponding to 2x up in Figure 8 ), a down-sampling layer (corresponding to 2x down in Figure 8 ), a first concatenation layer, a second concatenation layer, a third concatenation layer, a first convolution layer, a second convolution layer, an activation layer (corresponding to Sigmiod in Figure 8 ), a multiplication layer and a first addition layer.

[0041] The output of the up-sampling layer is connected with the input of the first concatenation layer, the output of the down-sampling layer is connected with the input of the second concatenation layer, the output of the first concatenation layer is connected with the input of the third concatenation layer through the first convolution layer, the output of the second concatenation layer is connected with the input of the third concatenation layer through the second convolution layer, the output of the third concatenation layer is connected with the input of the multiplication layer through the activation layer, the output of the multiplication layer is connected with the input of the first addition layer, and the output of the first addition layer is connected with the input of the feature fusion sub-network.

[0042] The up-sampling layer of the mth multi-scale adaptive fusion module is connected with the output of the m+2th FasterNet module, the down-sampling layer of the mth multi-scale adaptive fusion module is connected with the output of the mth FasterNet module, and the inputs of the first concatenation layer, the second concatenation layer, the multiplication layer and the first addition layer of the mth multi-scale adaptive fusion module are further connected with the output of the m+1th FasterNet module; m=1, 2,..., N-2; N represents the number of scale types of the feature maps extracted by the multi-scale feature extraction sub-network.

[0043] The present application constructs a multi-scale adaptive fusion module as shown in Figure 8 , which fully utilizes the shallow features and deep features at different levels in the multi-scale feature extraction sub-network. By adaptively fusing the detailed information in the shallow layer and the semantic information in the deep layer, the module not only effectively integrates the multi-scale feature information, but also reduces the loss of weak texture defects and tiny defects. The calculation formula of the mth multi-scale adaptive fusion module is shown in the following formula.

[0044] ;

[0045] wherein, , respectively represent the shallow feature map (feature map of the mth scale), the intermediate layer feature map (feature map of the m+1th scale) and the deep feature map (feature map of the m+2th scale) input to the multi-scale adaptive fusion module; and respectively represent the shallow feature map and the deep feature map after preliminary fusion; wherein, represents a down-sampling operation; represents an up-sampling operation; represents that the feature map is spliced in the feature channel; represents a convolution operation; represents that the feature map is spliced in the feature space; represents a Sigmoid activation function; represents a feature map and generated weight multiplication operation; represents a feature map element-wise addition operation; represents a feature map output by the multi-scale adaptive fusion module. Taking the four-scale feature maps shown in Figure 7 as an example, the multi-scale adaptive guiding sub-network is used to adaptively optimize the different level feature maps output by the multi-scale feature extraction sub-network, and the calculation formula is as follows:

[0046] ;

[0047] wherein, , , and represent four different scale feature maps output by the multi-scale feature extraction sub-network; represents a weighted addition operation of the feature map by the multi-scale adaptive fusion module; and represent two different scale adaptive fusion feature maps output by the multi-scale adaptive fusion module.

[0048] In another exemplary embodiment of the present application, as shown in Figure 5 , the feature fusion sub-network comprises: N-2 first splicing modules connected in sequence, a first full-dimensional dynamic feature extraction module (corresponding to ODFE Block1 in Figure 5 ) arranged between each adjacent two first splicing modules, three second full-dimensional dynamic feature extraction modules (corresponding to ODFE Block2, ODFE Block3 and ODFE Block4 in Figure 5The second stitching module is located between every two adjacent second full-dimensional dynamic feature extraction modules (ODFE Block 2) and the second stitching module is located between each two adjacent second full-dimensional dynamic feature extraction modules.

[0049] The output of the last first splicing module is connected to the input of the first second full-dimensional dynamic feature extraction module.

[0050] The inputs of the N-2 first stitching modules are connected one-to-one with the outputs of the N-2 multi-scale adaptive fusion modules.

[0051] The input of the first second concatenation module is also connected to the output of the first first full-dimensional dynamic feature extraction module; the input of the second second concatenation module and the input of the first first concatenation module are also connected to the output of the last FasterNet module in the multi-scale feature extraction sub-network.

[0052] The outputs of the three second full-dimensional dynamic feature extraction modules are connected one-to-one with the inputs of the three detector heads in the detector head sub-network.

[0053] A feature fusion subnetwork is used to accurately locate defects and perform splicing and fusion operations on feature maps of different resolutions output by a multi-scale adaptive guidance subnetwork. Specifically, this process encompasses both top-down and bottom-up splicing and fusion methods. Figure 5 The example shown is the splicing and fusion of the outputs of two multi-scale adaptive fusion modules. The corresponding calculation formulas are as follows:

[0054] ;

[0055] ;

[0056] In the formula, This represents the feature map at the Nth (N=4) scale output by the multi-scale feature extraction subnetwork; and These represent two feature maps at different scales output by the multi-scale adaptive guiding subnetwork. Feature maps are stitched together on feature channels; and These represent the first full-dimensional dynamic feature extraction module and the second full-dimensional dynamic feature extraction module, respectively. and These represent feature maps of two different scales output through top-down concatenation and fusion. for Figure 5 The output of ODFE Block1, for Figure 5 The output of the first ODFE Block2; and These represent feature maps of two different scales output by bottom-up splicing and fusion. for Figure 5 The output of the second ODFE Block2, This is the output of the third ODFE Block2.

[0057] In another exemplary embodiment of this application, such as Figure 9 As shown, Figure 9 (a) in the diagram describes the specific structure of the full-dimensional dynamic feature extraction module (referring to either the first or second full-dimensional dynamic feature extraction module). Taking the first full-dimensional dynamic feature extraction module as an example, the first full-dimensional dynamic feature extraction module includes: a first CBS layer, a second CBS layer, and a segmentation layer (corresponding to...). Figure 9 Split in the middle), the fourth splicing layer (corresponding to) Figure 9 The Concat module and multiple cascaded ODC Bottleneck modules. Figure 9 (b) shows the specific structure of the ODC Bottleneck module. Each ODC Bottleneck module includes a second additive layer and multiple full-dimensional dynamic convolutional layers; the multiple full-dimensional dynamic convolutional layers and the second additive layer are connected sequentially; the input of the second additive layer is also connected to the input of the first full-dimensional dynamic convolutional layer. Figure 9 As shown in (b), the full-dimensional dynamic convolutional layer uses batch normalized BN and the activation function SiLU.

[0058] The input of the first CBS layer is connected to the output of the first splicing module, which is connected to the input of the first full-dimensional dynamic feature extraction module.

[0059] The output of the first CBS layer is connected to the input of the segmentation layer. The output of the segmentation layer is connected to the input of the fourth concatenation layer and the input of the first full-dimensional dynamic convolutional layer of the first ODC Bottleneck module. The output of the second addition layer of each ODC Bottleneck module is also connected to the input of the fourth concatenation layer. The output of the fourth concatenation layer is connected to the input of the second CBS layer. The output of the second CBS layer is the output of the first full-dimensional dynamic feature extraction module to which it belongs.

[0060] This application constructs a kind of Figure 9 The full-dimensional dynamic feature extraction module shown (referring to the first or second full-dimensional dynamic feature extraction module) achieves more efficient feature extraction, and the calculation formula is as follows:

[0061]

[0062] In the formula, and respectively represent the input feature map and the output feature map of the full-dimensional dynamic feature extraction module; with representing the feature map output by the segmentation operation on the feature channel of the feature map; ) of the feature map; with respectively represent the input feature map and the output feature map of the preliminary extraction module (ODC Bottleneck module); ) represents the full-dimensional dynamic convolution operation; ODC_Neck() represents the execution operation of the ODC Bottleneck module. Conv() in the formula refers to Figure 9 “CBS, S = 1, K = 1” in, wherein S (stride) represents the stride of convolution, and K (kernal size) represents the size of the convolution kernel.

[0063] The present application combines the full-dimensional dynamic convolution with the feature fusion subnetwork, defines a new feature fusion subnetwork, which can be called a feature refinement fusion subnetwork. The sampling position of the convolution kernel of the full-dimensional dynamic convolution is no longer fixed, but is dynamically adjusted according to the characteristics of the input feature map, which can more effectively adapt to the feature extraction of complex surface defect structures, so that the feature fusion subnetwork can obtain more accurate defect position information. As shown in Figure 10 , the full-dimensional dynamic convolution formula is as follows:

[0064]

[0065] In the formula, the full-dimensional dynamic convolution introduces a multi-dimensional attention mechanism with a parallel strategy, which multiplies different attention weights along the position, channel, filter, and convolution kernel dimensions of the convolution, so that the convolution operation can adapt to the differences in each dimension of the input. Among them, and respectively represent the input feature map and the output feature map (having a height and a width of / channel) of the full-dimensional dynamic convolution; represents the attention scalar of the convolution kernel , i = 1, 2,..., n; n represents the number of dynamic convolution kernels; , and respectively represent three kinds of attention scalars along the spatial dimension, the input channel dimension, and the kernel space output channel dimension of the convolution kernel ; represents the multiplication operation along different dimensions of the kernel space. Figure 10 in, k represents the spatial position of k x k; is the convolution kernel ; represents a convolution filter.

[0066] Figure 10 It is described that in the full-dimensional dynamic convolution, how the feature map is point-multiplied with four different dimension generated attention weights. Figure 10 (a) to (d) in the above formula respectively represent that the feature map is multiplied with different weights at different positions of the convolution, the feature map is multiplied with different weights at different channels, the feature map is multiplied with different weights at different filters, and the feature map is multiplied with different weights at different convolution kernels.

[0067] In another exemplary embodiment of the present application, the surface defect detection method based on multi-scale adaptive guidance further comprises: a training process of the pre-trained surface defect detection network, specifically comprising:

[0068] (1) obtaining image samples of the object to be detected; each image sample corresponds to a real surface defect detection result.

[0069] (2) inputting the image samples of the object to be detected into the initial surface defect detection network to output surface defect detection sample prediction results.

[0070] (3) constructing a loss function and calculating a loss error according to the surface defect detection sample prediction results and the corresponding real surface defect detection results; the loss function comprises a bounding box regression loss term and a classification loss term; a modulation factor changing the weight coefficient is introduced in the bounding box regression loss term.

[0071] The present application defines a new bounding box regression loss function: FCIOU loss function, which introduces a modulation factor that can change the weight coefficient, gives a larger loss to the regression box with a higher IOU value, accelerates the convergence of the prediction box and improves the accuracy of the regression. The calculation formula is as follows:

[0072] ;

[0073] In the formula, represents the bounding box regression loss; represents the FCIOU loss function; represents the CIOU loss function; IOU is the intersection over union between the prediction box and the real box; the modulation factor is a hyperparameter with a value range of 0 to 1; the calculation formula of the CIOU loss function is as follows:

[0074] ;

[0075] In the formula, represents the center point of the prediction box and the center point of the real box The Euclidean distance between them; It is the diagonal distance between the smallest bounding rectangle of the predicted bounding box and the ground truth bounding box; and This represents the difference in aspect ratio between the predicted bounding box and the ground truth bounding box; IOU is the intersection-union ratio between the predicted bounding box and the ground truth bounding box, and its calculation formula is shown below:

[0076] ;

[0077] In the formula, and These represent the predicted bounding box and the ground truth bounding box, respectively.

[0078] The bounding box regression loss function and classification loss function are used to calculate the bounding box regression loss and classification loss between the target prediction result and the target's true annotation information obtained from the model. The formula for calculating the classification loss is shown below:

[0079] ;

[0080] In the formula, It is the sample size. It is the number of categories. Image samples Corresponding category The true label (0 or 1). It is a surface defect detection network for image samples Predicted as category The probability of.

[0081] Therefore, the expression for the loss function of the surface defect detection network is:

[0082] ;

[0083] In the formula, L represents the loss error; and It is the weight of the loss term.

[0084] (4) Adjust the initial surface defect detection network according to the backpropagation of the loss error until the loss error converges or the maximum number of training iterations is reached, and obtain the pre-trained surface defect detection network.

[0085] In the present application, a multi-scale feature extraction subnetwork is used to quickly extract coarse features from an input image to be detected; a multi-scale adaptive guidance subnetwork is constructed to adaptively optimize the multi-scale feature maps extracted; a feature fusion subnetwork is constructed to first accurately locate the defects of the different resolution feature maps obtained by the multi-scale adaptive guidance subnetwork, and then to splice and fuse them; then the spliced and fused feature maps are sent to a detection head subnetwork to obtain a target prediction result; a novel hybrid loss function is constructed to calculate the classification loss and the bounding box regression loss of the target prediction result and the target real label information obtained by the detection head, and based on the two types of losses, the model parameters of the surface defect detection network are adjusted through back propagation, and a surface defect detection network based on multi-scale adaptive guidance is obtained. The present application introduces a multi-scale adaptive guidance subnetwork between the multi-scale feature extraction subnetwork and the feature fusion subnetwork, reduces the loss of weak and small target information, and introduces a full-dimensional dynamic feature extraction module in the feature fusion subnetwork to enhance the feature extraction capability of multi-scale targets and complex shape targets, and constructs an improved bounding box regression loss function. The present application uses the multi-scale adaptive guidance subnetwork to fully utilize the shallow features and deep features at different levels in the multi-scale feature extraction subnetwork, reduces the loss of weak texture defects and small defects; further uses the feature fusion subnetwork to accurately adapt to the feature extraction of complex surface defect structures; finally, combined with the new hybrid loss function, the ability of the surface defect detection network to locate and identify defects is improved, the false detection rate and the missed detection rate of industrial surface defect detection are reduced, and the robustness of the model is improved.

[0086] The present application also provides an application scenario applying the above-mentioned surface defect detection method based on multi-scale adaptive guidance. Specifically: the surface defect detection method based on multi-scale adaptive guidance provided in the embodiment can be applied in the industrial steel surface defect detection scenario. The scenario includes an image acquisition link and a surface defect detection link; the image acquisition link is used to acquire defect images of the industrial steel to be detected; and the surface defect detection link is used to identify and locate the surface defects of the industrial steel to be detected according to the acquired defect images. The surface defect detection method based on multi-scale adaptive guidance provided in the embodiment belongs to the surface defect detection link.

[0087] In an exemplary embodiment, a computer device, which can be a server or a terminal, is provided, and an internal structure diagram of the computer device can be as shown in FIG. 1. Figure 11As shown in the figure. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capability. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the surface defect detection result of the to-be-detected object based on the surface defect detection network and the prediction. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with the terminal outside through the network connection. The computer program is executed by the processor to realize a surface defect detection method based on multi-scale adaptive guidance.

[0088] Those skilled in the art can understand that, Figure 11 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement. In one exemplary embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to realize the steps in each of the above method embodiments.

[0089] In one exemplary embodiment, a computer readable storage medium is provided, storing a computer program, which is executed by a processor to realize the steps in each of the above method embodiments.

[0090] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0091] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to a memory, a database or other medium used in the embodiments provided in the present application can include at least one of a non-volatile and a volatile memory. The non-volatile memory can include a read-only memory (ROM), a magnetic tape, a floppy disk, a flash memory, an optical storage, a high-density embedded non-volatile memory, a resistive random access memory (ReRAM), a magnetoresistive random access memory (MRAM), a ferroelectric random access memory (FRAM), a phase change memory (PCM), a graphene memory, etc. The volatile memory can include a random access memory (RAM) or an external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM), etc.

[0092] The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a blockchain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0093] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, they should be considered as the scope of the present application.

[0094] The principles and implementation modes of the present application are described by using specific examples in the present application. The above-mentioned embodiments are only used to help understand the method and its core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range can be changed. In conclusion, the content of the present application should not be understood as a limitation.

Claims

1. A method for surface defect detection based on multi-scale adaptive guidance, characterized in that, The method comprises: acquiring an image of a to-be-detected object; inputting the image of the to-be-detected object into a pre-trained surface defect detection network to predict a surface defect detection result of the to-be-detected object; the surface defect detection network comprises a multi-scale feature extraction sub-network, a multi-scale adaptive guidance sub-network, a feature fusion sub-network and a detection head sub-network connected in sequence; the multi-scale adaptive guidance sub-network is used for adaptively fusing a plurality of feature maps of different scales extracted by the multi-scale feature extraction sub-network to obtain adaptively fused feature maps of different scales; the multi-scale feature extraction sub-network comprises N FasterNet modules connected in series; an input of a first FasterNet module is the image of the to-be-detected object; each FasterNet module outputs a feature map of a different scale; the feature fusion sub-network is used for splicing and fusing the adaptively fused feature maps of different scales to obtain a spliced and fused feature map; and the detection head sub-network is used for outputting the surface defect detection result of the to-be-detected object according to the spliced and fused feature map; wherein the surface defect detection method based on multi-scale adaptive guidance further comprises a training process of the pre-trained surface defect detection network, specifically: acquiring image samples of to-be-detected objects; each image sample corresponds to a real surface defect detection result; inputting the image samples of the to-be-detected objects into an initial surface defect detection network to output surface defect detection sample prediction results; constructing a loss function and calculating a loss error according to the surface defect detection sample prediction results and the corresponding real surface defect detection results; the loss function comprises a bounding box regression loss term and a classification loss term; a modulation factor changing a weight coefficient is introduced in the bounding box regression loss term; adjusting the initial surface defect detection network according to the loss error by back propagation until the loss error converges or a maximum training iteration number is reached to obtain the pre-trained surface defect detection network; wherein an expression of the loss function is: ； wherein ; ; ； In the formula, L represents a loss error; represents a bounding box regression loss term; represents a classification loss term; IOU is an intersection over union between a predicted box and a real box; a modulation factor is a hyperparameter with a value ranging from 0 to 1; represents a Euclidean distance between a predicted box center point and a real box center point ; is a diagonal distance of a minimum circumscribed rectangle of a predicted box and a real box; represents a difference between aspect ratios of a predicted box; represents a difference between aspect ratios of a real box; M represents a sample number; is a class number, is an image sample , a real label of a corresponding class ; is a probability predicted by a surface defect detection network for the image sample to be a class ; and are loss term weights.

2. The method for surface defect detection based on multi-scale adaptive guidance according to claim 1, characterized in that, the multi-scale adaptive guidance sub-network comprises N-2 multi-scale adaptive fusion modules; each multi-scale adaptive fusion module comprises an up-sampling layer, a down-sampling layer, a first splicing layer, a second splicing layer, a third splicing layer, a first convolutional layer, a second convolutional layer, an activation layer, a multiplication layer and a first addition layer; an output of the up-sampling layer is connected to an input of the first splicing layer, an output of the down-sampling layer is connected to an input of the second splicing layer, an output of the first splicing layer is connected to an input of the third splicing layer through the first convolutional layer, an output of the second splicing layer is connected to an input of the third splicing layer through the second convolutional layer, an output of the third splicing layer is connected to an input of the multiplication layer through the activation layer, an output of the multiplication layer is connected to an input of the first addition layer, and an output of the first addition layer is connected to an input of the feature fusion sub-network; The up-sampling layer of the mth multi-scale adaptive fusion module is connected with the output of the m+2th FasterNet module, the down-sampling layer of the mth multi-scale adaptive fusion module is connected with the output of the mth FasterNet module, and the input of the first splicing layer, the second splicing layer, the multiplication layer and the first addition layer of the mth multi-scale adaptive fusion module is further connected with the output of the m+1th FasterNet module; m = 1, 2, …, N-2; N represents the number of scale types of the feature maps extracted by the multi-scale feature extraction sub-network.

3. The method of surface defect detection based on multi-scale adaptive guidance according to claim 2, characterized in that, The feature fusion sub-network comprises N-2 first splicing modules connected in sequence, N-2 first full-dimensional dynamic feature extraction modules arranged between every two adjacent first splicing modules, 3 second full-dimensional dynamic feature extraction modules connected in sequence, and 2 second splicing modules arranged between every two adjacent second full-dimensional dynamic feature extraction modules. The output of the last first splicing module is connected with the input of the first second full-dimensional dynamic feature extraction module. The inputs of the N-2 first splicing modules are respectively and correspondingly connected with the outputs of the N-2 multi-scale adaptive fusion modules. The input of the first second splicing module is further connected with the output of the first first full-dimensional dynamic feature extraction module; the input of the second second splicing module and the input of the first first splicing module are further connected with the output of the last FasterNet module in the multi-scale feature extraction sub-network. The outputs of the 3 second full-dimensional dynamic feature extraction modules are respectively and correspondingly connected with the inputs of 3 detection heads in the detection head sub-network.

4. The method of surface defect detection based on multi-scale adaptive guidance according to claim 3, characterized in that, The first full-dimensional dynamic feature extraction module comprises a first CBS layer, a second CBS layer, a segmentation layer, a fourth splicing layer and a plurality of ODC Bottleneck modules connected in series; each ODC Bottleneck module comprises a second addition layer and a plurality of full-dimensional dynamic convolution layers; the plurality of full-dimensional dynamic convolution layers and the second addition layer are connected in sequence; the input of the second addition layer is further connected with the input of the first full-dimensional dynamic convolution layer; The input of the first CBS layer is connected with the output of the first splicing module connected with the input end of the first full-dimensional dynamic feature extraction module; The output of the first CBS layer is connected with the input of the segmentation layer; the output of the segmentation layer is respectively connected with the input of the fourth splicing layer and the input of the first full-dimensional dynamic convolution layer of the first ODC Bottleneck module; the output of the second addition layer of each ODC Bottleneck module is further connected with the input of the fourth splicing layer; the output of the fourth splicing layer is connected with the input of the second CBS layer; and the output of the second CBS layer is the output of the first full-dimensional dynamic feature extraction module.

5. The method for surface defect detection based on multi-scale adaptive guidance according to claim 1, characterized in that, The image of the object to be detected is input into the pre-trained surface defect detection network to predict the surface defect detection result of the object to be detected, specifically comprising: The image of the object to be detected is input into the multi-scale feature extraction sub-network to extract a plurality of feature maps of different scales; The multi-scale adaptive fusion sub-network is used to adaptively fuse the plurality of feature maps of different scales to obtain adaptive fusion feature maps of different scales; and The adaptive fusion feature maps of different scales are input into the detection head sub-network to obtain the detection result of the object to be detected. The adaptive fusion feature of different scales is spliced and fused by using a feature fusion sub-network to obtain a spliced and fused feature map; The spliced and fused feature map is input into a detection head sub-network to output a surface defect detection result of the object to be detected.

6. A computer device comprising: A memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that the processor executes the computer program to implement the surface defect detection method based on multi-scale adaptive guidance according to any one of claims 1-5.

7. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the surface defect detection method based on multi-scale adaptive guidance according to any one of claims 1-5.

Images