A method for detecting the profile of a knitted upper fabric

Abstract

The application discloses a kind of knitted shoe face cloth profile detection methods, comprising the following steps: S1.data acquisition;S2. Main network is constructed;S3. Edge network is constructed;S4. The setting of loss function;S5. Training network model.This method can accurately locate edge information, and the precision of cloth profile edge detection is fine, and the accuracy is high.

Description

TECHNICAL FIELD

[0001] The present application relates to the contour edge detection of knitted shoe upper fabric image, belongs to the image processing field under the machine vision field of industrial application, and in particular to a contour detection method of knitted shoe upper fabric. BACKGROUND

[0002] The algorithm research of image edge detection is the basis of many advanced computer vision tasks, such as segmentation, image restoration, object detection, etc. With the development of deep learning in the field of machine vision applications, the difficulty of image edge detection technology focuses on how to detect a complete edge contour and how to make the detected edge more refined. Many researchers have also carried out a large number of academic researches around this problem, and the research of edge detection algorithm based on deep learning has also made certain progress. For example, the method [Xie S 2015] uses convolutional neural network for edge detection research, which is considered as a pioneering work of applying convolutional neural network to edge detection field. This method uses VGG16 as the backbone network to extract features of the input image, and supervises each stage of the backbone network through deep supervision. The edge network of the edge map is simply composed of the intermediate layer output of each stage of the backbone network, then reduced to a single channel gray image to get the predicted edge map (Xie S, Tu Z. Holistically-nested edge detection [C] / / Proceedings of the IEEE international conference on computer vision. 2015: 1395-1403.). Subsequently, some scholars also proposed some new edge detection networks based on the backbone network VGG16, such as RCF, BDCN, etc.Method [liu Y 2017] starts from the perspective of convolution, uses rich convolutional features to build a new edge detection network (Liu Y, Cheng M M, Hu X, et al. Richer convolutional features for edge detection [C] / / Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 3000-3009.); Method [He J 2019] starts from the scale of feature map, believes that each scale has different features to learn, and thus builds a new edge detection network through bidirectional cascade, which is better than all previous edge detection networks (He J, Zhang S, Yang M, et al. Bi-directional cascade network for perceptual edge detection [C] / / Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition. 2019: 3828-3837.); In addition, method [Su Z 2021] also provides a new idea, which integrates the gradient calculation operation in the traditional edge detection method into deep learning, so that the edge detection inference in deep learning is to calculate the gradient of the image, thereby improving the accuracy of edge detection (Su Z, Liu W, Yu Z, et al. Pixel difference networks for efficient edge detection [C] / / Proceedings of the IEEE / CVF International Conference on Computer Vision. 2021: 5117-5127.).

[0003] Defects of prior art:

[0004] The acquisition of cloth images with complex textures is often through industrial cameras and corresponding light sources. Since the quality of the acquired images is greatly related to the light source, whether the cloth image is clear depends on whether the light is good enough. The current contour edge detection methods have some defects. First, almost all edge detection methods are not end-to-end, and subsequent post-processing algorithms are needed to obtain better edge quality. Second, the current low-level feature maps do not have good semantic information, and the high-level network does not have important detail information. Third, with the deepening of the network depth, it is difficult to retain the detail information. In the process of detecting the contour edge of the knitted shoe upper cloth, the image has no color difference, so how to solve the above problems is crucial. SUMMARY

[0005] The purpose of the present application is to meet the time-consuming needs of industrial machine vision applications while detecting the contour edge of the cloth, and to provide a contour detection method for knitted shoe upper cloth. This method can accurately locate the edge information, and the contour edge detection of the cloth is accurate and precise.

[0006] The technical solution to achieve the purpose of the present application is:

[0007] A contour detection method for knitted shoe upper cloth, comprising the following steps:

[0008] S1. Data acquisition: The data set comes from two sources. The first type is the HED-BSDS data set provided by the method [Xie S 2015], and the second type is the image of the actual light shooting of the contour of the knitted shoe upper cloth under the industrial camera. The HED-BSDS data set provides images taken from various scenes, and the acquired images are processed as follows:

[0009] S11: Matching of image pairs: Since data acquisition needs to be one-to-one corresponding in the form of image-manual annotation image, both the image and the manual annotation image need to be processed as image pairs. The manual annotation image is obtained by manually annotating the image target through the tool lableme and extracting the image edge through the globalPb algorithm.

[0010] (1) The image and the manually annotated image are cropped to a size of 341*481;

[0011] (2) Through the python script, two files are generated respectively for the image and the artificial annotation image, one is the train.lst file, and the train.lst file contains the image path corresponding to the training image and the artificial annotation image; the other file is the test.lst file, and the content in the test.lst file is the same as that in the train.lst file, that is, the image paths are matched one by one, so that the image and the artificial annotation image form an image pair;

[0012] S12: Preparation of data operation function: including reading of data set path, normalization, reading and storage of data and image format, shuffling of order, cropping of size (cutting is to adapt to the memory needs of different graphics cards, and also to reduce the calculation amount and speed up the training) and judgment of parameter amount;

[0013] S2. Building a backbone network: the backbone network extracts the features of the input image, and the information extracted from the input image is the basis for subsequent visual tasks. The backbone network is mainly composed of the following steps:

[0014] S21: Building differential convolution: integrate the gradient operation in the edge detection of traditional images into deep learning. When convolving the image, first calculate the gradient of the pixels in the image region covered by the convolution kernel and save it. Then, the calculated gradient and the convolution kernel are convolved to complete the construction of differential convolution, and a convolution block is obtained;

[0015] S22: Building a feature extraction block: build a feature extraction block based on differential convolution. The feature extraction block extracts features from the image by stacking different numbers of convolution blocks to form a module. The number of convolution blocks in the feature extraction block is 4 blocks. Then, the feature extraction block is stacked to form a backbone network. The size of the image entering the residual layer of the backbone network is 341*481, the number of channels in the first stage is 60, the number of channels in the second stage is 120, the number of channels in the third stage is 240, and the number of channels in the fourth stage is 240. The feature channel outputs a feature map for the input image. If the feature map output by each stage is directly extracted as a whole feature, the calculation amount and parameter amount will be large, and the calculation formula of the calculation amount is:

[0016] MAC=C in *kernel_size*kernel_size*H out *W out *C out ,

[0017] In order to optimize the network, reduce the operation time and make the network model meet the actual application in the industrial field, the channel separation is performed on the feature channel in the feature extraction stage, and the separated channels are respectively convolved to reduce Cin To reduce MAC, according to the concept of lightweight network design, first, the feature channel separation of the image is performed using the depth separable operation, and then the point convolution operation is used to make the information interaction of each channel. Set group_numbers = in_channels, that is, the feature channels are divided into in_channels groups, that is, each group has only one channel, and then each convolution kernel is only responsible for feature extraction of one channel to produce in_channels feature maps, and then the information interaction of each channel is performed through point convolution, that is, each sub-feature map is output into a feature map through 1*1conv. Through the depth separable and point convolution operations, the calculation formula of the calculation amount is: MAC = C in *kernel size *kernel size *H in *W in +1*1*H out *W out *C1 in *C out It can be seen that compared with directly performing convolution on the feature channel to extract features, the depth separable operation reduces the number of parameters by group_numbers times, and speeds up the training and inference speed of the network model. In a conventional case, the residual layer is stacked in the ResNet manner, and the model degradation problem is solved by using a jump connection, which is represented as:

[0018] y = C(x) + x;

[0019] S23. Constructing a down-sampling module: down-sampling the image through convolution with a step of strides = 2;

[0020] S3. Building an edge network: the main function of the edge network is to infer the features extracted by the backbone network, which can also be said to be decoding, and then completing the corresponding visual task. The edge network is composed of a channel attention module, a spatial attention module, a multi-scale feature extraction module, an up-sampling module, a refinement module, a cross-layer weight distribution strategy and a channel dynamic division algorithm.

[0021] S31. Constructing a channel attention module: the purpose is to focus on the importance of the channel in the feature map, and highlight the importance of a certain channel through weighting, and then focus attention on important channels in subsequent convolution. The channel attention is reduced in size to 1*1 through global average pooling and global maximum pooling, and then the attention of the two pooling methods is added to obtain the final channel attention score. The formula of the channel attention is as follows:

[0022] F c = σ(Avg(f avg)+Avg(f max ));

[0023] S32. Constructing a spatial attention module: The spatial attention emphasizes more on which position in the space is more important, that is, which pixel is more important for the prediction of the result, which can also be called position attention. The spatial attention mechanism is set by first reducing the channel number of an in_channels number of channels to 4 channels through convolution for feature extraction and dimension reduction, then performing a convolution operation again to extract features and reduce the channel number to a single channel, and finally calculating the spatial attention score through a sigmoid function. The formula of the spatial attention is as follows:

[0024] F s (c=1)=σ(f H*W (c));

[0025] S33. Constructing a multi-scale feature extraction module: The multi-scale feature extraction module extracts features from the same feature map using convolution kernels with different hole rates, aiming to obtain semantic information of the feature map. The hole rates d used are d=3, d=5, d=7, d=9, and d=11. Finally, the feature maps extracted under different hole rates are added to obtain the final feature map to obtain semantic information. The formula of the multi-scale feature extraction is as follows:

[0026]

[0027] S34. Constructing an up-sampling module: Since the deconvolution operation realizes the size and channel conversion of the reconstructed image, the operation is simple but prone to uneven brightness caused by the convolution step size not being able to divide the convolution kernel size. This phenomenon is also called "checkerboard artifact". The up-sampling module technology uses bilinear interpolation and convolution operation to realize the conversion of the size and channel of the reconstructed image. The purpose of up-sampling is achieved by directly using bilinear interpolation to enlarge the image. After the interpolation algorithm, a normal convolution layer is used to realize the matching of the image feature channel. After repeating the operation twice, the reconstructed image size is converted to in_channe / 2 output. The idea of bilinear interpolation algorithm is as follows: four feature points of the original image are used to estimate a point of the reconstructed new image. In the x and y directions of the coordinates, three single linear interpolation operations are performed. If the new image label is to be calculated, the coordinates of the four points of the original image must be known. In the x-axis direction, two single linear interpolation operations are performed, that is, the coordinates of R1 are calculated using Q 11 , Q 12 , Q 21 , Q 22 two points, and the coordinates of R2 are calculated using Q 11 , Q 21 , Q 21 , Q22 Two points R2 coordinates:

[0028]

[0029]

[0030] The R1, R2 two point coordinates obtained are subjected to single linear interpolation operation to obtain the P point coordinates of the reconstructed image:

[0031]

[0032] The upsampling method adopted avoids the uneven overlapping phenomenon of the reconstructed image caused by the deconvolution operation, better meets the visual recognition experience from the visual effect, and plays an important role in the contour edge detection of the knitted shoe face fabric image, and provides basic information for the generation of the final edge map;

[0033] S35. A refinement module is proposed: since the final edge is thick and thick, the principle of the design of the refinement module is to remove linearity and introduce nonlinearity. First, the feature map is processed through two parallel routes, one of which introduces sub-pixel convolution to increase nonlinearity to disrupt the linearization of space, and the other increases nonlinearity through different activation functions to disrupt the excessive linear relationship in space. The formula of the refinement module is as follows:

[0034] F(x)=PS(W L *f L-1 (x LR )+b L )+σ(RELU(x));

[0035] S36. A cross-layer weight distribution strategy is proposed: by proposing a cross-layer weight distribution strategy, the low-level feature map has high-level semantic information, which is convenient for accurate positioning of edge details. First, a weight of each stage of the feature map is calculated, then a set of weights is obtained, and finally the set of weights is redistributed, that is, the weights calculated by the high-level feature map are distributed to the low-level feature map, so that the low-level feature map has semantic information and highlights the main information. The formula of the cross-layer weight distribution strategy is as follows:

[0036] F(x1, x2)=σ(x1)*x2+σ(x2)*x1;

[0037] S37. A channel dynamic division algorithm is proposed: in the contour edge detection of the knitted shoe upper cloth image, the texture details are very important, however, in the deep network, the detail information has been lost, in order to preserve the details, a channel dynamic division algorithm is proposed, first, the importance of the channel is calculated to find out the amount of information possessed by each channel, then a threshold is set to separate the channels with different information amount for grouping, finally, the different groups are processed with additional different weighting, the formula of the channel dynamic division algorithm is as follows:

[0038]

[0039] C=C1+C2+C3;

[0040] S4. Setting of loss function: in the training stage, the annotator-robust loss function in the method of [liu Y 2017] is adopted, and the loss function is derived from two parts:

[0041] (1) Annotator-robust loss function:

[0042]

[0043] (2) Dice loss function:

[0044]

[0045] The purpose of the Dice loss function is to find the feature mapping difference between the predicted edge image and the manually annotated edge image, and then minimize the difference to obtain fine edges, so the total loss is represented as:

[0046]

[0047] S5. Parameter registration, training network model: the specific settings of the training parameters are as follows:

[0048] batch_size=24、max_epoch=20、Optimizer=Adam、learning_rate=5*10 -3 、decay_step=10epoch patch_size=1200。

[0049] The innovation points of the technical solution in this edge network stage are summarized as:

[0050] (1) A cross-layer weight distribution strategy is proposed. It solves the problem of less semantic information in low-level feature maps and inaccurate positioning of detail information in high-level feature maps, which is very important for knitted fabric images with complex texture.

[0051] (2) A thinning module is proposed. It mainly solves the problem of too thick and too rough edges detected, which is very important for accurate edge information positioning. It can get more refined edges, and refined edges are crucial for applications.

[0052] (3) A channel dynamic division algorithm is proposed. It mainly solves the problem of difficult to retain details, which is essential for retaining more details for complex texture images, so as to more easily detect the contour edge of knitted shoe upper fabric.

[0053] (4) Improvement of loss function. By introducing the dice loss function, more refined edges can be obtained, and the best performance can be obtained by minimizing the loss function.

[0054] This method can accurately locate edge information, and the detection accuracy of fabric contour edge is fine and the accuracy is high. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 The flowchart of the embodiment;

[0056] Figure 2 The schematic diagram of the bilinear interpolation principle in the embodiment;

[0057] Figure 3 The principle diagram of the channel dynamic division algorithm in the embodiment;

[0058] Figure 4 The structure diagram of the thinning module in the embodiment;

[0059] Figure 5 The cross-layer weight distribution strategy diagram in the embodiment;

[0060] Figure 6 The detail diagram of the contour edge detection effect of knitted shoe upper fabric image in the embodiment;

[0061] Figure 7 The detail diagram of the contour edge detection effect of HED-BSDS data set in the embodiment. DETAILED DESCRIPTION

[0062] The invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, but not limited to the invention.

[0063] Embodiment:

[0064] Reference Figure 1 A knitted shoe upper fabric contour detection method, comprising the following steps:

[0065] S1. Data acquisition: There are two sources of data sets, the first is from the HED-BSDS data set provided by the method [Xie S 2015], and the second is the actual light shooting image of the profile of the knitted shoe upper fabric under the industrial camera. The HED-BSDS data set provides more than 20,000 images with a resolution of 341*481 taken from various scenes, and the acquired images are processed as follows:

[0066] S11: Matching of image pairs: Since data acquisition needs to be one-to-one corresponding in the form of image-manual annotation image, both image and manual annotation image data image sources need to be processed as image pairs. The manual annotation image is obtained by manually annotating the image target through the tool lableme and extracting the image edge through the globalPb algorithm;

[0067] (1) The image and the manually annotated image are cropped to 341*481 size;

[0068] (2) Two files are generated by python scripts for image and manually annotated image respectively, one is train.lst file, which contains the image paths of the training image and manually annotated image one-to-one corresponding, and the other file is test.lst file, which contains the same content as train.lst file, that is, the path corresponding image is one-to-one matched, so that the image and the manually annotated image form an image pair;

[0069] S12: Preparation of data operation function: including reading of data set path, normalization, reading and storage of data and image format, disorder of order, size of cutting (cutting is on one hand to adapt to the memory needs of different graphics cards, on the other hand is also to reduce the amount of calculation, speed up the training speed) and parameter quantity judgment;

[0070] S2. Building the main network: The main network extracts the features of the input image, and the information extracted from the input image is the basis for subsequent visual tasks. The main network is mainly composed of the following steps:

[0071] S21: Building differential convolution: The gradient operation in the edge detection of traditional image is integrated into deep learning. When the image is convolved, the gradient of the pixels in the image region covered by the convolution kernel is calculated and saved first, and then the solved gradient and the convolution kernel are convolved to complete the construction of differential convolution, and the convolution block is obtained;

[0072] S22: Constructing a feature extraction block: constructing a feature extraction block based on differential convolution, the feature extraction block extracts features from the image by stacking different numbers of convolution blocks to form a module, the number of convolution blocks in the feature extraction block is 4 blocks, then stacking the feature extraction blocks to form a backbone network, the size of the image entering the residual layer of the backbone network is 341*481, the number of feature channels in the first stage is 60, the number of feature channels in the second stage is 120, the number of feature channels in the third stage is 240, and the number of feature channels in the fourth stage is 240, the feature channel outputs a feature map for the input image, if the feature map output by each stage is directly subjected to overall feature extraction, the calculation amount and parameter amount are large, wherein the calculation formula of the calculation amount is:

[0073] MAC = C in *kernel_size*kernel_size*H out *W out *C out ,

[0074] To optimize the network, reduce the operation time and make the network model meet the actual application in the industrial field, the feature channels are separated in the feature extraction stage and the separated channels are respectively subjected to convolution to reduce C in to reduce MAC, according to the design principle of lightweight network, first, the feature channels of the image are separated by using depth separable operation, then the information of each channel is interacted by using point convolution operation, set group_numbers = in_channels, that is, the feature channels are divided into in_channels groups, that is, each group has only one channel, then each convolution kernel is only responsible for extracting features of one channel to generate in_channels feature maps, then the information of each channel is interacted by point convolution, that is, 1*1conv is used to output each sub-feature map into a feature map again, through the depth separable and point convolution operations, the calculation formula of the calculation amount is: MAC = C in *kernel size *kernel size *H in *W in +1*1*H out *W out *C1 in *C out It can be seen that compared with directly extracting features by convolution on the feature channels, the parameter amount is reduced by group_numbers times by using depth separable operation, and the training and inference speed of the network model is accelerated, and in the conventional case, the residual layer is stacked in the ResNet manner, and the model degradation problem is solved by using the jump connection manner, which is represented as:

[0075] y = C(x) + x;

[0076] S23. Constructing a down-sampling module: down-sampling the image through convolution with a step strides = 2;

[0077] S3. Constructing an edge network: the main role of the edge network is to infer the features extracted by the backbone network, which can also be said to be decoding, and then completing the corresponding visual task, the edge network is composed of a channel attention module, a spatial attention module, a multi-scale feature extraction module, an up-sampling module, a refinement module, a cross-layer weight distribution strategy and a channel dynamic division algorithm;

[0078] S31. Constructing a channel attention module: the purpose is to focus on the importance of channels in the feature map, and highlight the importance of a channel through weighting, and then focus attention on important channels in subsequent convolution, in this example, the channel attention is reduced in size to 1*1 through global average pooling and global maximum pooling, and then the attention of the two pooling methods is added to obtain the final channel attention score, the formula of the channel attention is as follows:

[0079] F c = σ (Avg (f avg ) + Avg (f max ) ) ;

[0080] S32. Constructing a spatial attention module: spatial attention emphasizes more on which position in the space is more important, that is, which pixel is more important for the prediction of the result, which can also be called position attention, in this example, the spatial attention mechanism is set first through convolution to reduce the channel number from in_channels to 4 channels, and then perform a convolution operation to extract features and reduce the channel number to a single channel, and finally calculate the spatial attention score through the sigmoid function, the formula of the spatial attention is as follows:

[0081] F s (c = 1) = σ (f H*W (c) ) ;

[0082] S33. Constructing a multi-scale feature extraction module: the multi-scale feature extraction module in this example extracts features from the same feature map using convolution kernels with different hole rates, aiming to obtain semantic information of the feature map, in this example, the hole rates d used are d = 3, d = 5, d = 7, d = 9, d = 11; Finally, the feature maps extracted under different hole rates are added to obtain the final feature map to obtain semantic information, the formula of the multi-scale feature extraction is as follows:

[0083]

[0084] S34. Constructing an up-sampling module: since the deconvolution operation realizes the size and channel conversion of the reconstructed image, the operation is simple but prone to uneven light and dark phenomenon caused by the fact that the convolution step cannot be evenly divided by the size of the convolution kernel, which is also called "checkerboard artifact". The up-sampling module technology realizes the conversion of the size and channel of the reconstructed image by using bilinear interpolation and convolution operation. The purpose of up-sampling is achieved by directly using bilinear interpolation to enlarge the image. After the interpolation algorithm, a normal convolution layer is used to realize the matching of the image feature channel. After repeating the operation twice, the size of the reconstructed image is converted to in_channe / 2 output. The idea of bilinear interpolation algorithm is as follows: four feature points of the original image are used to estimate a point of the new reconstructed image. In the x and y directions of the coordinates, three single linear interpolation operations are performed, as shown in Figure 2 . If the new image label is to be calculated, the coordinates of the four points Q 11 , Q 12 , Q 21 , and Q 22 of the original image must be known. In the x-axis, two single linear interpolation operations are performed, i.e. Q 11 and Q 21 are used to calculate the R1 coordinate, and Q 21 and Q 22 are used to calculate the R2 coordinate:

[0085]

[0086]

[0087] The R1 and R2 point coordinates obtained are subjected to single linear interpolation operation to obtain the P point coordinates of the reconstructed image:

[0088]

[0089] The up-sampling method adopted avoids the uneven overlapping phenomenon of the reconstructed image caused by the deconvolution operation, which better meets the visual recognition experience in terms of visual effect and plays an important role in the contour edge detection of the knitted shoe upper fabric image, providing basic information for the generation of the final edge map.

[0090] S35. Proposing a thinning module: since the final edge is thick and rough, it is hoped that a relatively thin edge can be extracted, so the principle of the design of the thinning module is to remove linearity and introduce nonlinearity. First, the feature map is processed through two parallel routes. One route introduces sub-pixel convolution to increase nonlinearity to disrupt the linearization of space, and the other route uses different activation functions to increase nonlinearity to disrupt the excessive linear relationship in space, as shown in Figure 4 . The formula of the thinning module is as follows:

[0091] F(x) = PS(W L *f L-1 (x LR )+b L )+σ(RELU(x));

[0092] S36. A cross-layer weight distribution strategy is proposed: by proposing a cross-layer weight distribution strategy, the low-level feature map has high-level semantic information, which facilitates accurate positioning of edge details. First, calculate the weight of each stage of the feature map, then derive a set of weights, and finally redistribute this set of weights, i.e. the weights calculated by the high-level feature map are distributed to the low-level feature map, so that the low-level feature map has semantic information and highlights the main information. The formula of the cross-layer weight distribution strategy is as follows, as shown in Figure 5

[0093] F(x1, x2) = σ(x1) * x2 + σ(x2) * x1;

[0094] S37. A channel dynamic division algorithm is proposed: in the contour edge detection of knitted shoe fabric images, texture details are crucial, but in deep networks, detailed information has been lost. In order to preserve details, a channel dynamic division algorithm is proposed. First, calculate the weight of the feature map to find out the amount of information each channel has, then set a threshold to separate channels with different amounts of information for grouping, and finally perform additional different weighting processing on these different groups. The formula of the channel dynamic division algorithm is as follows, as shown in Figure 3

[0095] C = C1 + C2 + C3;

[0096] S4. Setting of loss function: in the training stage, the annotator-robust loss function in the method of [liu Y 2017] is adopted. The loss function comes from two parts:

[0097] (1) Annotator-robust loss function:

[0098]

[0099] (2) Dice loss function:

[0100]

[0101]

[0102] ​​The purpose of the Dice loss function is to find the feature mapping difference between the predicted edge image and the artificially labeled edge image, and then minimize this difference to obtain a fine edge. The experiment found that minimizing the two loss functions can take their respective advantages, so the total loss is represented as:

[0103]

[0104] S5. Parameter registration, training network model: The specific settings of the training parameters are as follows:

[0105] batch_size = 24, max_epoch = 20, Optimizer = Adam, learning_rate = 5*10 -3 , decay_step = 10epoch patch_size = 1200,

[0106] The specific settings of the experimental environment in this example are shown in Table 1:

[0107] Table 1 Experimental environment parameter table

[0108]

[0109] Figure 6 The profile edge detection effect detail map of the knitted shoe fabric image, through the method of this example, it can be seen that in Figure 6 (a), most of the knitted shoe fabric image background does not exist color difference, edge is not exist, only through the global interpretation can be inferred that there is a contour edge. The current edge detection method, from the traditional method to the deep learning method, cannot detect the contour edge of the knitted shoe fabric image which does not exist obvious edge, complex texture and little color difference. The method of this example can accurately detect the contour.

[0110] Figure 7 The profile edge detection effect detail map of the HED-BSDS dataset, it can be seen that the method of this example can detect the contour edge of the fabric image, and can also detect the edge of the image on the public dataset, has good generalization, in Figure 7 (c), it can be seen that the low-level feature map of the edge detection network contains more semantic information, and the high-level feature map retains more detailed information, and the edge detection network model can well extract the contour edge of the image.

Claims

1. A method of profile detection of a knitted shoe upper fabric, characterized in that, Comprise the following steps: S1. Data acquisition: there are two sources of data sets, the first is HED-BSDS data set, the second is the actual light shooting image of the profile of knitted shoe upper fabric under industrial camera, HED-BSDS data set provides images taken from various scenes, the acquired images are processed as follows: S11: matching of image pairs: since data acquisition needs to be in the form of image-manual annotation image one-to-one correspondence, the image and manual annotation image data image source needs to be processed into image pairs, the manual annotation image is obtained by manually marking the image target through the tool lableme and extracting the image edge through the globalPb algorithm; (1) the image and the manual annotation image are cut to 341*481 size; (2) two files are generated by python script for image and manual annotation image respectively, one is train.lst file, train.lst file contains the image paths of training image and manual annotation image one-to-one correspondence, the other file is test.lst file, the content of test.lst file is the same as that of train.lst file, that is, the image paths are matched one by one, so that the image and the manual annotation image form an image pair; S12: preparation of data operation function: including reading of data set path, normalization, reading and storage of data and image format, disorder of order, size cutting and parameter quantity judgment; S2. Constructing the main network: the main network extracts the features of the input image, and the information extracted from the input image is the basis for the subsequent visual tasks, the main network is composed of the following steps: S21: constructing differential convolution: the gradient operation in the edge detection of traditional image is integrated into deep learning, the gradient of the pixels in the image region covered by the convolution kernel is calculated and saved when the image is convolved, then the calculated gradient and the convolution kernel are convolved to complete the construction of differential convolution, and the convolution block is obtained; S22: constructing a feature extraction block: constructing a feature extraction block based on differential convolution, the feature extraction block extracts features of an image by stacking a convolution block in different numbers to form a module, the number of convolution blocks in the feature extraction block is 4 blocks, then stacking the feature extraction blocks forms a backbone network, the size of the image entering the residual layer of the backbone network is , the first stage of feature channels , the second stage , the third stage , the fourth stage , the feature channel outputs a feature map for the input image, if the feature map output by each stage is directly subjected to overall feature extraction, the calculation amount and the parameter amount are large, and the calculation formula of the calculation amount is: ， In order to optimize the network, reduce the operation time and make the network model meet the actual application in the industrial field, the feature channels are separated and the separated channels are respectively convolved in the feature extraction stage, so as to reduce The calculation amount is reduced According to the design principle of the lightweight network, firstly, the feature channel separation of the image is performed by using the depth separable operation, then the information interaction of each channel is performed by using the point convolution operation, and it is set That is, the feature channels are divided into in_channels groups, that is, each group has only one channel, and then each convolution kernel is only responsible for feature extraction of one channel to generate in_channels feature maps, and then the information interaction of each channel is performed by using the point convolution, that is, the Each sub-feature map is output as a feature map again, and the calculation amount is calculated by using the depth separable and point convolution operations, and the calculation formula is: ， By the depth separable operation, the parameter quantity is less times, accelerates the training and inference speed of the network model, and the residual layer under the conventional condition is stacked in the ResNet manner, and the model degradation problem is solved by using the jump connection manner, which is expressed as: ； S23. Constructing down-sampling module: down-sampling the image through convolution with strides=2; S3. Constructing edge network: the edge network is composed of channel attention module, spatial attention module, multi-scale feature extraction module, up-sampling module, thinning module, cross-layer weight distribution strategy and channel dynamic division algorithm; S31. Constructing channel attention module: channel attention reduces the size to 1*1 through global average pooling and global maximum pooling, then adds the attention of the two pooling methods to get the final channel attention score, the formula of channel attention is as follows: ； S32. Constructing spatial attention module: the spatial attention mechanism is set by first reducing the channel number of in_channels to 4 channels through convolution, then performing convolution operation again to extract features and reduce the channel number to a single channel, and finally calculating the spatial attention score through sigmoid function, the formula of spatial attention is as follows: ； S33. Constructing a multi-scale feature extraction module: the multi-scale feature extraction module extracts features from the same feature map using convolution kernels with different hole rates, aiming to obtain semantic information of the feature map. The adopted hole rates d are d=3, d=5, d=7, d=9, and d=11. Finally, the feature maps extracted under different hole rates are added to obtain the final feature map to obtain semantic information. The formula of multi-scale feature extraction is as follows: ； S34. Constructing the up-sampling module: since the deconvolution operation realizes the size and channel conversion of the reconstructed image, the operation is simple but prone to uneven light and dark phenomenon caused by the convolution step length not being able to divide the convolution kernel size, which is also called "checkerboard artifact", the up-sampling module technology realizes the size and channel conversion of the reconstructed image by using bilinear interpolation and convolution operation, the purpose of up-sampling is achieved by directly using bilinear interpolation to enlarge the image, and then using a normal convolution layer to realize the matching of the image feature channel after the interpolation algorithm, after repeating the operation twice, the size of the reconstructed image is converted to in_channe / 2 output, the idea of bilinear interpolation algorithm is: four feature points of the original image are used to estimate a point of the reconstructed new image, three single linear interpolation operations are performed in the x and y directions of the coordinates, if the new image is to be calculated, the coordinates of the four points of the original image must be known, twice single linear interpolation operation is performed on the axis, that is, the coordinates of two points are used to calculate the coordinates of two points ​​​​​​​ ， ， The obtained The two-point coordinates are subjected to a single linear interpolation operation to obtain the Point coordinates: ； S35. Proposing a thinning module: since the final edge is thick and thick, the principle of designing the thinning module is to remove linearity and introduce nonlinearity. First, the feature map is processed through two parallel routes. One route introduces sub-pixel convolution to increase nonlinearity to disrupt the linearization of space. The other route uses different activation functions to increase nonlinearity to disrupt the excessive linear relationship in space. The formula of the thinning module is as follows: ； S36. Proposing a cross-layer weight distribution strategy: by proposing a cross-layer weight distribution strategy, the low-level feature map has high-level semantic information, which facilitates accurate positioning of edge details. First, the weight of each stage of the feature map is calculated, and then a set of weights is obtained. Finally, this set of weights is redistributed, i.e., the weights calculated by the high-level feature map are distributed to the low-level feature map, so that the low-level feature map has semantic information and highlights the main information. The formula of the cross-layer weight distribution strategy is as follows: ； S37. Proposing a channel dynamic division algorithm: in the contour edge detection of knitted shoe fabric images, texture details are crucial. However, in deep networks, detailed information has been lost. To preserve details, a channel dynamic division algorithm is proposed. First, the weight of the feature map is calculated to find out the amount of information each channel has. Then, a threshold is set to separate channels with different amounts of information for grouping. Finally, these different groups are processed with additional different weights. The formula of the channel dynamic division algorithm is as follows: ， ； S4. Setting of loss function: in the training stage, the annotator-robust loss function in the method of [liu Y 2017] is adopted. The loss function is derived from two parts: (1) Annotator-robust loss function: ， (2) Dice loss function: ， The purpose of the Dice loss function is to find the feature mapping difference between the predicted edge image and the artificially labeled edge image, and then minimize this difference to obtain fine edges. Therefore, the total loss is represented as: ; S5. Parameter registration and training of network model: the specific settings of the training parameters are as follows: 。

Images