A method and device for detecting defects of light-weight sanitary products

Abstract

The present application relates to a method for detecting defects of light-weight sanitary products, which method comprises: acquiring a surface image of the sanitary product and inputting it into a light-weight target detection neural network model based on deep learning to perform data enhancement processing on the surface image, thereby obtaining an initial feature map, performing multi-scale feature extraction on the initial feature map through a CSP module embedded with an EVC module, fusing feature maps of different scales through a feature pyramid method of an FPV module, fusing feature maps of different resolutions through a path aggregation method of a PANet module, and fusing the output features of the CSP module, the FPV module and the PANet module through a light-weight convolution operation of a CSConv module to obtain and output a final detection result. By integrating the GSConv and EVC modules into the deep learning target detection neural network model, the accuracy and efficiency of target detection are improved while maintaining the characteristics of light weight and speed.

Description

TECHNICAL FIELD

[0001] The present application relates to a method for detecting defects of light-weight sanitary products, belonging to the technical field of computer vision and deep learning. BACKGROUND

[0002] In manufacturing industry, surface defects are important issues in product production and quality control. Traditional surface defect detection methods mainly rely on manual visual inspection, which is time-consuming, labor-intensive and subjective. In recent years, with the continuous development and application of deep learning technology, surface defect detection methods based on deep learning have gradually become a research hotspot. This method realizes automatic detection and classification of different types of surface defects through self-learning and feature extraction of surface defect images, with high precision and efficiency. However, existing surface defect detection methods based on deep learning generally have too many model parameters, large computational complexity and low running efficiency, which cannot meet the requirements of real-time performance and response speed in production and manufacturing scenarios.

[0003] Currently, traditional target detection algorithms usually use methods based on sliding window or region proposal for target detection. However, this method needs to densely sample the entire image, resulting in huge computational complexity. At the same time, the method based on region proposal also has many problems, such as non-maximum suppression, inaccurate proposal, etc., which limits the performance of target detection. In addition, most traditional target detection algorithms use shallow networks as backbone networks, which limits their understanding and expression ability of target semantic information, further affecting their detection performance. Therefore, these problems make it difficult for traditional target detection algorithms to achieve efficient and accurate target detection in complex scenarios.

[0004] In summary, existing surface defect detection methods based on deep learning have too many model parameters, large computational complexity and low running efficiency. In order to solve these problems, new methods and technologies need to be adopted. SUMMARY

[0005] The present application provides a method for detecting defects of light-weight sanitary products, aiming to at least solve one of the technical problems existing in the prior art.

[0006] The method for detecting defects of light-weight sanitary products of the present application technical solution, the method comprises the following steps:

[0007] S110, acquiring the surface image of the sanitary product and inputting it into the light-weight target detection neural network model based on deep learning to perform data preprocessing on the surface image, and then obtaining an initial feature map;

[0008] Among them, the target detection neural network model includes a CSP module, an FPV module, a PANet module and a CSConv module.

[0009] S200. Multi-scale feature extraction is performed on the initial feature map through the CSP module embedded with the EVC module;

[0010] S300: The FPV module fuses feature maps of different scales, and the PANet module fuses feature maps of different resolutions.

[0011] S400: The output features of the CSP module, the FPV module, and the PANet module are fused through the lightweight convolution operation of the CSConv module to obtain and output the final detection result.

[0012] Furthermore, the EVC module includes a STEM block, an MLP block, and a visual center mechanism; step S200 includes:

[0013] S310. The input features of the EVC module are smoothed using the STEM block; wherein the STEM block includes a 7×7 first convolution.

[0014] S320. Obtain the output features of the STEM block, and capture the global long-term dependencies of the top features through the lightweight MLP block to obtain global information;

[0015] S330. Obtain the output features of the STEM block, and aggregate the local region features within the layer through the learnable visual center mechanism to obtain local information;

[0016] S340. Connect the result feature map of global information and the result feature map of local information together along the channel dimension to obtain the output feature of the ENC module;

[0017] The MLP block is connected in parallel with the visual center mechanism.

[0018] Furthermore, the output characteristics of the ENC module are calculated using the following formula:

[0019] X = cat(MLP(X) (in) ); LVC(X (in) ))

[0020] In the formula, X represents the output feature of the EVC module, Cat(·) represents the feature map stitching along the channel dimension, and MLP(X) represents the output feature of the EVC module. in ) represents the feature output of the MLP block, LVC(X) in X represents the output feature of the visual center mechanism. in The output characteristics of the STEM block are represented by the following formula:

[0021] X in =σ(BN(Conv) 7×7 (X4)))

[0022] In the formula, Conv 7×7 (·) represents a 7×7 convolution function with a stride of 1, BN(·) represents the batch normalization function, and σ(·) represents the ReLU activation function.

[0023] Furthermore, the MLP block includes a first remaining module and a second remaining module, and step S320 includes:

[0024] S321. Input the output features of the STEM block into the first residual module based on depthwise convolution, and simultaneously perform grouping and normalization processing; wherein, the output features of the first residual module are calculated using the following formula.

[0025]

[0026] In the formula, GN(·) represents the group normalization function, and DConv(·) represents the depthwise convolution function with a kernel size of 1x1;

[0027] S322. Input the processed output features of the first residual module into the second residual module based on channel MLP to perform signal scaling and DropPath operations; wherein, the output features MLP(X) of the second residual module are calculated using the following formula. in ):

[0028]

[0029] In the formula, CMLP(·) represents the function of the channel MLP.

[0030] Furthermore, the visual center mechanism includes a group of convolutional layers and a CBR block, and step S330 includes:

[0031] S331. The output features of the STEM block are encoded by the convolutional layer group, wherein the convolutional layer group includes a 1×1 second convolution, a 3×3 third convolution and a 1×1 fourth convolution;

[0032] S332. The encoded features are input into the CBR block for processing, wherein the CBR block includes 3BN convolutional layers and ReLU activation layers;

[0033] S333. Input the processed features into the codebook;

[0034] The output features of the visual center mechanism are obtained through the output features X of the STEM block.in and the local corner region feature Z, which is calculated as follows:

[0035]

[0036] where Z represents the local corner region feature; represents the channel-wise addition;

[0037] where the local corner region feature Z is calculated as follows:

[0038]

[0039] where Conv 1x1 represents a 1x1 convolution function, and δ(·) is a sigmoid function, is a channel-wise multiplication, and e represents the overall information of the entire image with respect to K codewords.

[0040] Further, the CSConv module includes a Ghost sub-block and a Shrink sub-block, and the step S400 includes:

[0041] S410, performing a channel dimension reduction operation on the input feature map based on the Ghost sub-block based on grouped convolution, to obtain a smaller Ghost feature map;

[0042] where the channel dimension reduction operation includes: dividing the input feature map into a plurality of groups, and then randomly selecting channels in each group to obtain the smaller Ghost feature map;

[0043] S420, performing a lightweight convolution operation on the Ghost feature map based on the Shrink sub-block based on depth separable convolution, to obtain a final feature map;

[0044] where the lightweight convolution operation includes: performing a depth convolution operation on each input channel respectively, and then performing a point-by-point convolution on the obtained outputs, to obtain the final feature map.

[0045] Further, the loss function of the model is based on a weighted non-maximum suppression algorithm, and the model loss function is calculated as follows:

[0046] L CIoU = 1 - IoU + R CIoU

[0047] where L CIoU represents the model loss function, IoU represents a bounding box loss function, R CIoU represents a penalty term;

[0048] wherein the penalty term is obtained according to minimizing the normalized distance between the centers of the two bounding boxes, and the penalty term R CIoU is calculated by the following formula:

[0049]

[0050] wherein b and bgt represent the centers of B and B gt , and p(·) represents the Euclidean distance, c represents the diagonal length of the minimum bounding box covering the two boxes. Alpha represents a trade-off parameter, and v represents the consistency of the aspect ratio.

[0051] Further, when training the model, a structured pruning operation is performed on the model; the structured pruning operation is based on weights or channels and includes the following steps:

[0052] S510, pruning a specific layer, channel or weight of the network model, removing redundant connections and parameters;

[0053] S520, retraining the pruned model to refit the training data and achieve a similar accuracy to the original model.

[0054] The technical scheme of the present application also relates to a computer device comprising a memory and a processor, wherein the processor executes a computer program stored in the memory to implement the above method.

[0055] The technical scheme of the present application also relates to a computer readable storage medium having computer program instructions stored thereon, wherein the computer program instructions are executed by a processor to implement the above method.

[0056] The beneficial effects of the present application are as follows.

[0057] The embodiment of the present application proposes a light-weight sanitary product defect detection method and system based on deep learning technology. The system uses a light-weight neural network structure and end-to-end learning technology to realize fast detection and classification of surface defects. Optimizing the model structure and parameters can reduce the amount of calculation, improve the model running efficiency and response speed, and realize real-time defect detection and quality control. At the same time, using effective feature extraction methods and classifiers, the system can realize fast detection and classification of different types of surface defects, and improve the detection accuracy and efficiency. In order to meet the real-time detection requirements of the sanitary product defect detection platform, the present application introduces the GSConv technology to reduce the model complexity and maintain the accuracy. In addition, the present application also embeds the EVC module to improve the accuracy of small target sample detection, and through the channel pruning technology, the size and calculation amount of the model are greatly reduced without losing the accuracy of the model, so as to improve the running efficiency of the model and accelerate the inference speed.

[0058] In addition, additional aspects and advantages of the present application will be set forth in part in the description that follows, and in part will be obvious from the description, or can be learned by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 is an architecture schematic diagram of a target detection neural network model according to an embodiment of the present application.

[0060] Figure 2 is an architecture schematic diagram of an EVC module according to an embodiment of the present application.

[0061] Figure 3 is an architecture schematic diagram of a CSConv module according to an embodiment of the present application. DETAILED DESCRIPTION

[0062] The concept, specific structure and generated technical effects of the present application will be described clearly and completely in the following combined with embodiments and drawings, so as to fully understand the purpose, scheme and effect of the present application.

[0063] It should be noted that, unless otherwise specified, when a certain feature is referred to as being "fixed", "connected" to another feature, it can be directly fixed, connected to the other feature, or indirectly fixed, connected to the other feature. The singular forms "a", "an" and "the" used in this text are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, unless otherwise defined, all technical and scientific terms used in this text have the same meaning as understood by those skilled in the art. The terms used in the specification of this text are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The term "and / or" used in this text includes any combination of one or more related listed items.

[0064] It should be understood that although the terms first, second, third, etc. can be used in this disclosure to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish one type of element from another. For example, without departing from the scope of the present disclosure, a first element can also be referred to as a second element, and similarly, a second element can also be referred to as a first element. The use of any and all examples or exemplary language provided herein (e.g., "for example", "for instance", etc.) is intended merely to better illustrate the embodiments of the present application and should not be interpreted as imposing a limitation on the scope of the present application unless otherwise required.

[0065] Reference Figures 1 to 3 The method for detecting defects of light-weight sanitary products according to the present application at least comprises the following steps:

[0066] S110, a surface image of the sanitary product is acquired and input into a lightweight target detection neural network model based on deep learning to perform data enhancement processing on the surface image, and then an initial feature map is obtained;

[0067] The target detection neural network model comprises a CSP module, an FPV module, a PANet module and a CSConv module;

[0068] S200, multi-scale feature extraction is performed on the initial feature map through the CSP module embedded with the EVC module;

[0069] S300, the feature maps of different scales are fused through the FPV module, and the feature maps of different resolutions are fused through the PANet module;

[0070] S400, the output features of the CSP module, the FPV module and the PANet module are fused through the lightweight convolution operation of the CSConv module to obtain and output the final detection result.

[0071] Referring to Figure 1 , the embodiment of the application aims to develop a target detection neural network model based on the YOLOv5 network model of deep learning to convert the target detection task into a regression problem. The basic framework of the model is divided into four parts: input, backbone, neck and prediction. In the input part, mosaic data enhancement is used to enhance the data set to improve the generalization ability of the model. The input data is subjected to various data enhancement processing, such as random cropping, color disturbance, deformation, etc., to increase the sample size of the data set, thereby improving the robustness and precision of the model. The backbone part adopts the CSP module to perform feature extraction, wherein the CSP module is a lightweight feature extraction module that can realize efficient multi-scale feature extraction. The CSP module performs feature extraction through CSPDarknet53 to improve the precision and speed of the model. The neck part mainly consists of two modules of FPN and PANet, which are used to aggregate image features at different scales. The FPN module fuses feature maps of different scales through the method of feature pyramid, thereby improving the detection ability of the model for small targets. The PANet module adopts the path aggregation method to aggregate feature maps of different resolutions together to improve the representation ability of the model. The prediction part fuses the outputs of the backbone network and the neck network through convolution operation to obtain the final detection result.

[0072] The model framework of the embodiment of the application adopts the GE-YOLOv5 framework (see Figure 1), to improve the accuracy and efficiency of target detection. The model is improved on the basis of YOLOv5. In the feature extraction part, an EVC module is embedded in the network to improve the detection ability of small targets and the effect of multi-scale feature fusion. In the feature fusion part, a lightweight GSConv convolution is used instead of a standard convolution to reduce the model parameter quantity and calculation quantity.

[0073] Detailed implementation of step S200

[0074] In some embodiments of the present application, in the feature extraction part, an EVC module is embedded in the CSP model. The EVC module is based on an explicit learnable visual center module, which can improve the small target detection accuracy, see Figure 2 The EVC module captures long-range dependencies through a lightweight multilayer perceptron (MLP) and aggregates local corner regions of the input image through a parallel learnable visual center mechanism, so that the EVC module extracts more comprehensive and discriminative feature representations without increasing the computational complexity, thereby effectively improving the detection accuracy of small targets.

[0075] Further, the EVC module is composed of two parallel connected sub-blocks, one of which is a lightweight MLP block, which is used to capture global long-term dependencies of top-level features (i.e. global information); the other sub-block is a learnable visual center mechanism, which is used to aggregate local region features within a layer to preserve local corner regions (i.e. local information). The resulting feature maps of the two sub-blocks are connected together along the channel dimension as the output of the EVC, which is used for downstream recognition.

[0076] Further, the input features of the EVC module are first processed by a STEM block for feature smoothing, to ensure that the feature mapping input to the EVC is smooth. The STEM block is composed of a 7x7 convolution with an output channel size of 256, followed by a batch normalization layer and an activation function layer. Through the STEM block, the EVC module can extract more comprehensive and discriminative features without increasing the computational complexity, which can effectively improve the accuracy of small target detection.

[0077] In an embodiment, the feature extraction of the embodiment of the present application includes the following steps:

[0078] S310, performing feature smoothing processing on the input features of the EVC module through a STEM block; wherein the STEM block includes a first 7x7 convolution;

[0079] S320, obtaining the output features of the STEM block, capturing the global long-term dependencies of top-level features through a lightweight MLP block to obtain global information;

[0080] S330, obtaining an output feature of the STEM block, and aggregating local region features in a learnable visual center mechanism aggregation layer to obtain local information;

[0081] S340, connecting the result feature map of the global information and the result feature map of the local information together along a channel dimension to obtain an output feature of the ENC module;

[0082] The MLP block is connected in parallel with the visual center mechanism.

[0083] Further, the output feature of the ENC module is calculated by the following formula:

[0084] X = cat (MLP (X (in) ); LVC (X (in) ))

[0085] In the formula, X represents the output feature of the EVC module, cat(·) represents the feature map splicing along the channel dimension, MLP(X in ) represents the feature output of the MLP block, LVC(X in ) represents the output feature of the visual center mechanism, and X in represents the output feature of the STEM block, wherein the output feature of the STEM block is calculated by the following formula:

[0086] X in = σ (BN (Conv 7×7 (X4)))

[0087] In the formula, Conv 7×7 (·) represents a 7x7 convolution function with a stride of 1, BN(·) represents a batch normalization function, and σ(·) represents a ReLU activation function.

[0088] In an application embodiment, the EVC module includes a lightweight MLP block, which is composed of two residual modules, including a first residual module based on deep convolution and a second residual module based on channel MLP, wherein the input of the second residual module is the output of the first residual module. The second residual module performs channel scaling operation and DropPath operation (a normalization method for preventing overfitting) on the input feature to improve the generalization ability and robustness of the feature. As for the first residual module based on deep convolution, the feature output by the STEM module X in is first input to a deep convolution layer, and group normalization processing is performed at the same time. Compared with the traditional spatial convolution, the deep convolution adopted in the method can not only reduce the calculation cost, but also improve the feature representation ability, and after channel scaling and downlink path, residual connection of X in is realized.

[0089] Specifically, the output features of the STEM block are input into the first residual module based on depthwise convolution, and grouping and normalization processing is performed simultaneously; wherein, the output features of the first residual module... The calculation is as follows:

[0090]

[0091] In the formula, GN(·) represents the group normalization function, and DConv(·) represents the depthwise convolution function with a kernel size of 1x1;

[0092] Specifically, the processed output features of the first residual module are input into the second residual module based on channel MLP for signal scaling and DropPath operations; the output features of the second residual module are calculated as follows:

[0093]

[0094] In the formula, CMLP(·) represents the function of the channel MLP.

[0095] In one application embodiment, the EVC module includes a Learnable Visual Center (LVC) mechanism. LVC is an encoder with an intrinsic dictionary, consisting of two components: 1) an intrinsic codebook B = {b1, b2, ..., bK}, where N = H × W is the total number of input features, and H and W are the height and width of the feature map, respectively; 2) a set of learnable visual center scaling factors S = {s1, s2, ..., sK}.

[0096] Specifically, the visual center mechanism originates from the STEM block X in The extracted features are first encoded using a group of convolutional layers, consisting of a 1×1 second convolution, a 3×3 third convolution, and a 1×1 fourth convolution. After encoding, the features are processed by a CBR (Computer Build Report) block, which includes 3BN convolutional layers and a ReLU activation function. Through these steps, the encoded feature X is obtained. in It was input into the codebook.

[0097] Among them, the output features of the visual center mechanism are obtained through the output features X of the STEM block. in The local angular region feature Z is obtained by channel addition; the local angular region feature Z is calculated as follows:

[0098]

[0099] In the formula, Conv 1x1denotes a 1x1 convolution function, and δ(·) is a sigmoid function. is the multiplication of channels, and e denotes the overall information of the entire image about K codebooks.

[0100] It should be noted that in the embodiments of the present application, a set of scale factors s is used to sequentially make and b k map to the corresponding position information. The information of the entire image about the kth codebook can be calculated in the following manner:

[0101]

[0102] In the formula, is the ith pixel point, b k is the kth learnable visual codebook, s k is the kth scale factor. is the information of each pixel position relative to the codebook. K is the total number of visual centers. Then e k is used to fuse all e , which contains a BN layer, a ReLU layer and a mean layer. On this basis, the overall information of the entire image about K codebooks is calculated as follows:

[0103]

[0104] After obtaining the output of the codebook, it is further fed into a fully connected layer and a 1x1 convolution layer to predict the features of the highlighted key class. After that, the input features from the STEM block X in are multiplied by the channel between the scale factor coefficient δ(·). Finally, the channel addition is performed between the output features X in of the STEM block and the local corner region features Z.

[0105] The method of the embodiments of the present application realizes intra-layer feature adjustment by embedding the EVC module, which can extract global long-range dependencies and also preserve the local corner region information of the input image as much as possible, thereby being conducive to improving the speed and accuracy of the dense prediction task.

[0106] Specific implementation of step S400

[0107] In some specific embodiments of the present application, the output features of the CSP module, the FPV module and the PANet module are fused by the lightweight convolution operation of the CSConv module to obtain and output the final detection result. Referring to Figure 3 , the GSConv module is a convolution module based on group convolution and group attention mechanism, and the GSConv module includes a Ghost sub-block based on group convolution and a Shrink sub-block based on depth separable convolution, thereby effectively improving the feature extraction capability of the model.

[0108] Specifically, the GSConv module divides the input feature map into multiple groups, performs independent convolution operations on each group, and interacts information between different groups through a grouped attention mechanism. In the convolution operation, the convolution kernel parameters within each group are shared, while the convolution kernel parameters between different groups are independent. In this way, the GSConv module can enhance the multi-channel feature extraction capability of the model, and effectively reduce the model parameter quantity and computational complexity.

[0109] In an embodiment, the feature fusion of the embodiment of the present application includes the following steps:

[0110] S410, performing a channel dimension reduction operation on the input feature map through a Ghost sub-block based on grouped convolution to obtain a smaller Ghost feature map. The channel dimension reduction operation includes dividing the input feature map into several groups, and then randomly selecting the channels within each group to obtain a smaller Ghost feature map.

[0111] S420, performing a lightweight convolution operation on the Ghost feature map through a Shrink sub-block based on depth separable convolution to obtain a final feature map. The lightweight convolution operation includes performing a depth convolution operation on each channel of the input respectively, and then performing a point-by-point convolution on the obtained output to obtain the final feature map.

[0112] In an application embodiment, the Ghost sub-block in the GSConv module is based on grouped convolution, which is used to improve the computational efficiency in the network. The Ghost sub-block reduces the amount of calculation and the amount of parameters by reducing the dimension of the channels of the input feature map. Specifically, the Ghost part divides the input feature map into several groups, and then randomly selects the channels within each group to obtain a smaller Ghost feature map. This Ghost feature map will be passed to the next Shrink sub-block.

[0113] In an application embodiment, the Shrink sub-block in the GSConv module is based on a depthwise separable convolution (DWConv) for further compression of the Ghost feature map. Specifically, the Shrink part adopts DWConv to perform depthwise convolution operation on the Ghost feature map, thereby further reducing the channel number and calculation amount. DWConv is based on a lightweight convolution operation, which is divided into two steps of depthwise convolution and pointwise convolution, i.e., depthwise convolution operation is performed on each channel of the input, and then the obtained output is subjected to pointwise convolution to obtain the final feature map. By adopting depthwise separable convolution, the calculation amount and parameter amount are greatly reduced while ensuring the model accuracy. The feature map output after the processing of the Shrink sub-block has a smaller channel number and less parameter amount, thereby making the entire network more lightweight and efficient.

[0114] It should be noted that the GSConv module of the embodiment of the present application realizes channel dimension reduction and compression of the calculation amount of the feature map by performing grouping convolution and depthwise convolution and other operations on the input feature map, which is beneficial to greatly reduce the parameter amount and calculation amount of the network while ensuring the model accuracy, so that the entire network is more lightweight and efficient. The GSConv module tries to preserve the connection of the feature map as much as possible, but if it is used in all stages of the model, the network layers of the model will be deeper, and the deep layer will increase the resistance to data flow, significantly increasing the inference time, so when these feature maps go to the Neck, they have become slender (the channel dimension reaches the maximum, and the width and height dimensions reach the minimum), and no longer need to be transformed, therefore, the model of the embodiment of the present application only uses the GSConv module in the Neck stage.

[0115] In some specific embodiments of the present application, when the target detection neural network model is trained, in prediction, a generalized IoU (GIoU) loss is used as a bounding box loss function, and a weighted non-maximum suppression algorithm (NMS) is used for NMS. The loss function is as follows:

[0116]

[0117]

[0118] where C is the minimum area size covering B and B gt . B gt =(x gt ,y gt ,w gt ,h gt ) is a label, and B=(x,y,w,h) is a predicted box.

[0119] However, when the prediction box is located within the label and the size of the prediction box is the same, it is impossible to distinguish the prediction box and the label, and therefore in some embodiments of the present application, the GIoU is replaced by a complete IoU (CIoU) loss. On the basis of the GIoU loss, the CIoU loss considers the overlapping area, the distance between the center points of the bounding boxes, and the consistency of the aspect ratio of the bounding boxes. The loss function can be defined as:

[0120]

[0121]

[0122] L CIoU =1-IoU+R CIoU

[0123] where R CIoU is a penalty term, which is defined by minimizing the normalized distance between the center points of the two bounding boxes. b and bgt represent the center points of B and B gt , ρ(·) is the Euclidean distance, c is the length of the diagonal of the minimum bounding box covering the two boxes. α is a positive trade-off parameter, and v measures the consistency of the aspect ratio.

[0124] And the trade-off parameter α is defined as:

[0125]

[0126] where the overlapping area factor is given higher regression priority, especially for non-overlapping cases.

[0127] In an application embodiment, the method of the embodiments of the present application adopts sparse network pruning to optimize the GE-YOLOv5 model, so as to reduce the size and computational overhead of the model while maintaining a certain accuracy through model pruning, and improve the practicability of the model. The model pruning of the method of the present application successfully reduces the number of parameters of the model to 3 / 16 of the original, and maintains similar performance and accuracy to the original model.

[0128] When training the model, the present application adopts a structured pruning operation as a lightweight method for convolutional neural networks. The advantage of the structured pruning method is that it is easy to implement and deploy in software and hardware, and can effectively reduce the complexity of the model without affecting the accuracy of the network.

[0129] Specifically, the structured pruning method of the present application includes two steps of pruning and retraining. In the pruning stage, specific layers, channels or weights in the network are pruned to remove redundant connections and parameters. In the retraining stage, the pruned model is retrained to refit the training data and achieve similar accuracy to the original model. The structured pruning method is based on the structural characteristics of the deep neural network, and adopts a targeted pruning strategy to remove unnecessary parameters in the network without affecting the integrity of the network structure and the feature expression ability. The method is based on weight or channel pruning, which can compress and accelerate the model.

[0130] Further, different pruning strategies can be used for different network layers, such as channel-based, weight-based or filter-based pruning. The structured pruning method can effectively reduce the complexity of the model, thereby improving the training speed and inference speed of the model, and also reducing the storage space of the model.

[0131] Specifically, the weight or channel-based pruning operation of the present application includes the following steps:

[0132] S510, pruning specific layers, channels or weights of the network model to remove redundant connections and parameters;

[0133] S520, retraining the pruned model to refit the training data and achieve similar accuracy to the original model.

[0134] Further, the present application uses a structured pruning method, and the introduction of the BN layer with learnable parameters γ and β in the neural network can accelerate the training and convergence speed of the network. The channel data is normalized by translation and scaling to learn the feature distribution of the network. The L1 regularization term is added to the loss function to reduce the complexity of the model and obtain a sparse network. In the present application method, a scale factor is introduced in each channel of the BN layer, and a penalty term about γ is added to the loss function to obtain a sparse network. The first term of the loss function represents the loss function of normal network training, the second term represents the L1 regularization term, λ represents the balance factor of the two terms, and Γ represents the set of all pruned channels. Finally, the size of the scale factor γ of the sparse network serves as an indicator of the importance of each layer of the network channel. The structured pruning method used in the present application method is more convenient in software and hardware implementation and deployment.

[0135] Further, on the basis of the lightweight benchmark network, through iterative sparse training, the gamma parameter of the BN layer in the neural network will gradually decrease, and the overall distribution tends to 0, realizing the sparsification of the network, which is beneficial to channel screening and pruning. By adjusting the size of the hyperparameter lambda, the network can obtain appropriate sparsity, thereby screening redundant channels. Removing channels with smaller contributions in the network will not affect the effective extraction of features by the model, and can also reduce the complexity of the network. Re-training and fine-tuning can improve the accuracy and generalization performance of the model. The single threshold control channel method will make the number of channels of the neural network lose the original regular structure, therefore, by limiting the number of channels of the network, the channels are reorganized using a mask during channel selection to meet the requirements of model deployment acceleration, and a new lightweight network model reconstruction is realized.

[0136] The method for detecting defects of lightweight sanitary products has multiple advantages, such as high precision, fast operation, real-time performance, universality and high scalability, and can provide technical support for automatic defect detection and quality control of the manufacturing industry. The method can improve the accuracy and efficiency of target detection while maintaining the characteristics of lightweight and fast, making it suitable for large-scale practical application scenarios.

[0137] It should be appreciated that the method steps in the embodiments of the present application can be realized or implemented by computer hardware, a combination of hardware and software, or through computer instructions stored in a non-transitory computer readable memory. The method can use standard programming techniques. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if necessary, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, the program can run on a programmed special-purpose integrated circuit for this purpose.

[0138] In addition, the operations of the processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and / or combinations thereof) can be performed under the control of one or more computer systems configured with executable instructions (e.g., executable instructions, one or more computer programs or one or more applications), and can be implemented as code (e.g., executable instructions, one or more computer programs or one or more applications) executing collectively on one or more processors, by hardware or a combination thereof. The computer programs include a plurality of instructions executable by one or more processors.

[0139] Further, the methods can be implemented in any type of computing platform operably connected to a suitable computing platform, including but not limited to a personal computer, mini-computer, mainframe, workstation, network or distributed computing environment, separate or integrated computer platforms, or in communication with charged particle tools or other imaging devices, and the like. Aspects of the present application can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated to the computing platform, such as a hard disk, optical read and / or write storage media, RAM, ROM, and the like, such that it can be read by a programmable computer to configure and operate the computer to perform the processes described herein when the storage medium or device is read by the computer. In addition, the machine-readable code, or portions thereof, can be transmitted over wired or wireless networks. The present application described herein includes these and other different types of non-transitory computer readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The present application can also include the computer itself when programmed in accordance with the methods and techniques described herein.

[0140] The computer program can be applied to input data to perform the functions described herein to transform the input data to generate output data that is stored to non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present application, the transformed data represents a physical and tangible object, including a particular visual depiction of the physical and tangible object produced on a display.

[0141] The above description is only preferred embodiments of the present application, and the present application is not limited to the above-described embodiments, but any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the present application. The technical solutions and / or embodiments of the present application can have various modifications and changes within the scope of the present application.

Claims

1. A method for defect detection of lightweight sanitary products, the method comprising the following steps: S110, acquiring a surface image of a sanitary product and inputting it into a lightweight target detection neural network model based on deep learning to perform data preprocessing on the surface image, thereby obtaining an initial feature map; wherein the target detection neural network model comprises a CSP module, an FPV module, a PANet module and a CSConv module; S200, in the backbone part, performing multi-scale feature extraction on the initial feature map through the CSP module embedded with an EVC module; S300, fusing feature maps of different scales through the FPV module and fusing feature maps of different resolutions through the PANet module; S400, fusing the output features of the CSP module, the FPV module and the PANet module through the lightweight convolution operation of the CSConv module to obtain and output the final detection result; wherein the EVC module comprises a STEM block, an MLP block and a visual center mechanism; the step S200 comprises: S310, performing feature smoothing processing on the input features of the EVC module through the STEM block; wherein the STEM block comprises a first convolution of 7x7; S320, obtaining the output features of the STEM block, capturing the global long-term dependency relationship of the top-level features through the lightweight MLP block to obtain global information; S330, obtaining the output features of the STEM block, aggregating the local area features within the layer through the learnable visual center mechanism to obtain local information; S340, connecting the result feature map of the global information and the result feature map of the local information together along the channel dimension to obtain the output features of the EVC module; wherein the MLP block and the visual center mechanism are connected in parallel; wherein the output features of the EVC module are calculated by the following formula: X = cat(MLP(X (in) ); LVC(X (in) )) where X denotes the output feature of the EVC module, Cat(·) denotes the feature map concatenation along the channel dimension, MLP(X in ) denotes the feature output of the MLP block, LVC(X in ) denotes the output feature of the visual center mechanism, and X in denotes the output feature of the STEM block, wherein the output feature of the STEM block is calculated by the following formula: X in = σ(BN(Conv 7×7 (X4))) where Conv(·) denotes a 7x7 convolution function with a stride of 1, BN(·) denotes a batch normalization function, and σ(·) denotes a ReLU activation function. 7×7 (·) denotes a 7x7 convolution function with a stride of 1, BN(·) denotes a batch normalization function, and σ(·) denotes a ReLU activation function. wherein the visual center mechanism comprises a convolution layer group and a CBR block, and the step S330 comprises: S331, encoding the output features of the STEM block through the convolution layer group, wherein the convolution layer group comprises a second convolution of 1x1, a third convolution of 3x3 and a fourth convolution of 1x1; S332, inputting the encoded features into the CBR block for processing, wherein the CBR block comprises a 3BN layer convolution and a ReLU activation layer; S333, inputting the processed features into a codebook; where the output features of the visual center mechanism are obtained by channel addition between the output features X of the STEM block in and the local angular region features Z, which are computed as follows: In the formula, Z represents a local angular region feature; represents a channel-wise addition. wherein the local angular region feature Z is calculated as follows: where Conv 1x1 denotes a 1x1 convolution function, δ(·) is a sigmoid function, is a multiplication of channels, e represents the overall information of the entire image with respect to the K codewords.

2. The method of claim 1, wherein, The MLP block comprises a first residual module and a second residual module, and the step S320 comprises: S321, input the output feature of the STEM block into the first residual module based on deep convolution, and perform grouped normalization processing function; wherein the output feature of the first residual module is calculated by the following formula wherein GN(·) represents group normalization processing and DConv(·) represents a depth convolution with a kernel size of 1x1; S322. Input the processed output features of the first residual module into the second residual module based on channel MLP to perform signal scaling and DropPath operations; wherein, the output features MLP(X) of the second residual module are calculated using the following formula. in ): wherein CMLP(·) represents the function of channel MLP.

3. The method of claim 1, wherein, The CSConv module comprises a Ghost sub-block and a Shrink sub-block, and the step S400 comprises: S410, performing a channel dimension reduction operation on the input feature map by the Ghost sub-block based on a grouped convolution to obtain a smaller Ghost feature map; The channel dimension reduction operation comprises: dividing the input feature map into a plurality of groups, and then randomly selecting channels in each group to obtain the smaller Ghost feature map; S420, performing a lightweight convolution operation on the Ghost feature map by the Shrink sub-block based on a depth separable convolution to obtain a final feature map; The lightweight convolution operation comprises: performing a depth convolution operation on each channel of the input respectively, and then performing a point-by-point convolution on the obtained output to obtain the final feature map.

4. The method of claim 1, wherein, The loss function of the model is based on a weighted non-maximum suppression algorithm, and the model loss function is calculated as follows: L CIoU = 1 - IoU + R CIoU In the formula, L CIoU denotes a model loss function, and IoU denotes a bounding box loss function, R CIoU denotes a penalty term; wherein the penalty term is obtained according to minimizing the normalized distance between the centers of the two bounding boxes, the penalty term R CIoU is calculated by the following formula: where b and bgt denote the center points of B and B gt , p(·) denotes the Euclidean distance, c denotes the diagonal length of the minimum enclosing box covering both boxes, a denotes a trade-off parameter, and v denotes a measure of consistency of the aspect ratio.

5. The method of claim 1, wherein, When training the model, a structured pruning operation is performed on the model; the structured pruning operation is based on weights or channels, and comprises the following steps: S510, pruning a specific layer, channel or weight of the network model to remove redundant connections and parameters; S520, retraining the pruned model to make it fit the training data again and achieve a similar accuracy to the original model.

6. A computer apparatus comprising a memory and a processor, characterised in that, The processor executes the computer program stored in the memory to implement the method of any one of claims 1-5.

7. A computer readable storage medium having program instructions stored thereon, the program instructions being executed by a processor to implement the method of any one of claims 1-5.

Images