A method and system for online detection of foreign fibers in cotton

Abstract

The present invention proposes an online detection method and system for foreign fibers in cotton, the steps of which are: a polarization camera collects images of the detection area of ​​the cotton irradiated by the laser in real time; the collected images constitute sample data, and the sample data is enhanced to obtain an enhanced sample set; the sample set is pre-trained using a migration method to obtain a pre-classified image; the pre-classified image is identified and classified using a parallel cascade deep learning method to obtain the type and position of the foreign fiber; the identified and classified image is histogram-equalized using a CLAHE algorithm to obtain the position of the foreign fiber with a target detection frame; the foreign fiber ROI algorithm is used to extract the foreign fiber position of the target detection frame to obtain the geometric parameters of the foreign fiber target. The present invention realizes the integration of optics, mechanics, electronics and computers, and can achieve ultra-high accuracy detection for various foreign fibers, with an accuracy rate of more than 98%; by combining two parallel convolutional neural networks, online preparation for classification and identification of multiple types of foreign fibers can be achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of cotton foreign fiber detection, and in particular to an online detection method and system for cotton foreign fiber, which integrates photoelectric detection technology, intelligent vision technology, parallel cascade deep learning and migration algorithm to realize online detection of foreign fibers contained in cotton and real-time positioning of identified foreign fibers. Background Art

[0002] Cotton is the largest natural fiber in the global fiber, textile and clothing economy. my country has a huge cotton production and consumption. The cotton textile industry consumes nearly 40% of the global cotton resources, ranking first in the world. China is a major cotton producer and consumer, and the cotton industry occupies an important position in my country's national economy. Cotton is one of the important raw materials for textile enterprises to process. During the picking, acquisition and processing, a large amount of foreign fibers are mixed in due to various factors. Foreign fibers in cotton refer to non-cotton fibers and non-natural fibers mixed in cotton, such as cotton shells, cotton seeds, plastics, hair, chemical fibers, paper pieces, etc. These foreign fibers mixed in raw cotton are easily broken into scattered single fibers, which are difficult to remove in the subsequent textile processing, and will have a serious adverse effect on the quality of textile products. These foreign fibers will not only reduce spinning efficiency, but also cause defects on the cloth surface and reduce product grades. Therefore, the detection and removal of foreign fibers in cotton is an important and necessary link before cotton processing. In recent years, many new foreign fiber detection methods have emerged, and the main basis used includes photoelectric detection technology, machine vision technology, ultrasonic technology, etc. However, these technologies are difficult to apply alone to actual production.

[0003] Photoelectric detection technology can only detect foreign fibers with color. Ultrasonic technology can only detect foreign fibers with a larger area and at a slower speed. General machine vision technology is not ideal for detecting foreign fibers with similar colors and small foreign fibers in cotton fibers. In addition, the on-site production environment will be affected by factors such as light, dust, and vibration. The complexity of the environment will seriously affect the accuracy of the detection. There are many types of foreign fibers. It is difficult for a single existing technology to detect all types of foreign fibers. The combination of multiple different technologies brings difficulties to the on-site detection of foreign fibers due to various problems such as instrument size and cost. The above methods are difficult to detect cotton-like fibers such as transparent plastics and wool, so there is an urgent need for a method that can achieve accurate measurement of these two types of foreign fibers. Summary of the invention

[0004] In view of the technical problem that existing detection systems or methods are difficult to detect transparent plastics and cotton-like fibers such as wool, and have poor real-time performance, the present invention proposes an online detection method and system for cotton foreign fibers, which can achieve accurate measurement of both transparent plastic and wool foreign fibers with good real-time performance.

[0005] In order to achieve the above object, the technical solution of the present invention is implemented as follows: a cotton foreign fiber online detection method, the steps are as follows:

[0006] Step 1: The polarization camera collects images of the detection area of ​​the cotton irradiated by the laser in real time;

[0007] Step 2: The collected images form sample data, and data enhancement is performed on the sample data to obtain an enhanced sample set;

[0008] Step 3: Use transfer learning method to pre-train the sample set to obtain pre-classified images;

[0009] Step 4: The parallel cascade deep learning method is trained through the pre-classified images to obtain a training model. The training model recognizes and classifies the images collected by the polarization camera in step 1 to obtain the type and location of foreign fibers.

[0010] Step 5: Perform histogram equalization on the image after identification and classification through the CLAHE algorithm to obtain the position of the foreign fiber with the target detection frame;

[0011] Step 6: The foreign fiber ROI algorithm extracts the foreign fiber position in the target detection frame and obtains the geometric parameters of the foreign fiber target.

[0012] The parallel cascade deep learning method includes a parallel cascade of an improved YOLOv5 algorithm and a fluorescent effect foreign fiber target detection algorithm, wherein the improved YOLOv5 algorithm is used to identify foreign fibers with obvious classification color features in an image, and the fluorescent effect foreign fiber target detection algorithm is used to identify foreign fibers that produce a fluorescent effect under laser irradiation; the improved YOLOv5 algorithm is trained using pre-classified images to obtain a trained YOLOv5 model, and the fluorescent effect foreign fiber target detection algorithm is trained using the pre-classified images to obtain a fluorescent effect foreign fiber target detection model; in actual detection, the YOLOv5 model processes images captured in real time by a polarization camera, and the images detected by the fluorescent effect foreign fiber target detection model are images of the original images captured by the polarization camera after being grayscaled.

[0013] The data enhancement method includes at least two of image mirroring, brightness change, Gaussian blur, pixel translation, size scaling, rotation transformation and / or pixel discarding.

[0014] The fluorescent effect foreign fiber target detection algorithm includes a feature extraction part and a regression prediction part. The feature extraction part includes 6 sequentially connected convolutional layers for extracting deep features of the image; the regression prediction part includes an FCN structure, and the FCN structure includes 3 sequentially connected convolutional layers. The deep features and shallow features are combined in the regression prediction part through 3 sequentially connected convolutional layers for convolution processing, and then up-sampling is performed. The up-sampled features and shallow features are spliced ​​to generate fused features, and regression prediction is performed using the fused features to obtain a prediction matrix.

[0015] The input end of the detection network of the improved YOLOv5 algorithm processes the pre-classified image through Mosaic data enhancement, the backbone network of the detection network is a Focus structure and a CSP structure connected in sequence, the detection network adopts an improved Mish activation function, and the network layer of the detection network adopts an FPN+PAN structure; the output end of the detection network includes a bounding box loss function and NMS non-maximum suppression; the depth coefficient depth_multiple of the improved YOLOv5 algorithm is 0.33, and the width coefficient width_multiple is 0.33.

[0016] The improved Mish activation function is:

[0017]

[0018]

[0019] Where x is the independent variable, ω=4(x+1)+4e 2x +e 3x +e x (4x+6), and δ=2e x +e 2x +2.

[0020] The bounding box loss function adopts the CIOU Loss loss function, and:

[0021]

[0022] Where d = ρ (A, B) is the relationship between the B frame and the B gt The Euclidean distance of the coordinates of the center points of the boxes, and c is the diagonal distance of the smallest box that encloses them; α is the weight function, v is used to measure the consistency of the aspect ratio, and:

[0023]

[0024]

[0025]

[0026] The final CIoU Loss loss function is:

[0027]

[0028] The improved YOLOv5 algorithm uses an adaptive picture scaling method to process images and an adaptive anchor frame calculation, that is, by calculating the difference between the predicted frame and the real frame, the network parameters are updated in reverse and iterated;

[0029] The NMS non-maximum suppression adopts DIOU-NMS.

[0030] A detection system for an online detection method of foreign fibers in cotton comprises a conveyor belt, a bracket is arranged above the conveyor belt, a polarization camera and a laser are mounted on the bracket, the light emitting area of ​​the laser is consistent with the image acquisition area of ​​the polarization camera, and the polarization camera and the laser are both connected to a computer. The computer processes the image acquired by the polarization camera through an online detection method, extracts foreign fiber targets and outputs the geometric parameters of the foreign fiber targets.

[0031] The laser is a 405nm blue-violet laser, and the sensor of the polarization camera is equipped with four-way pixel-level polarization filters of 0°, 45°, 90°, and 135°; the bracket is provided with a height adjustment mechanism, and the polarization camera and the laser are both fixed on the height adjustment mechanism; the height adjustment mechanism includes a lifting rod, and the lifting rod is provided with a positioning column, and the positioning column matches the positioning hole on the bracket.

[0032] The polarization camera and the laser are fixed on the bracket at an angle of 45° in the horizontal direction.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] 1. The online detection system can quickly and accurately identify and locate foreign fibers contained in cotton based on photoelectric detection, intelligent vision, parallel cascade deep learning and migration algorithms. At the same time, combined with the moving speed of the cotton flow, it can also locate the identified foreign fibers in real time, and help the foreign fiber removal device placed behind the identification and detection device to accurately remove the detected foreign fibers, thereby improving the quality of raw cotton and cotton textiles.

[0035] 2. The online detection system realizes the integration of optics, mechanics, electronics and computers, and can detect various foreign fibers, such as mulch, chemical fibers, cotton stalks, cotton leaves, paper pieces, etc. with ultra-high accuracy, and its detection accuracy can reach more than 98%.

[0036] 3. The blue-violet laser is irradiated at an angle of 45° to the foreign fibers containing fluorescent substances in cotton. The images collected by the polarization camera can show obvious characteristics. The foreign fiber target detection algorithm using the fluorescence effect can achieve the real-time detection requirements of white transparent foreign fibers such as mulch. The use of polarization cameras can minimize the interference of ambient light and reduce the eigenvalues ​​in the image, so that better detection effects can be achieved using fewer convolution kernels. In view of this feature, the traditional YOLOv5 algorithm was improved to reduce the number of convolution kernels in its network structure by half. Through testing, the improved YOLOv5 algorithm has reduced the volume of the trained model by 53%, increased the detection speed by 41%, and the detection time for a single image can reach 0.01s / image at the fastest, that is, in theory, it can achieve real-time detection of cotton flow moving at a speed of 0.4m / s.

[0037] 4. The transfer learning method for data imbalance balances the data ratio, prevents the model from non-convergence caused by imbalanced sample ratios, changes the three-classification task into two two-classification tasks, and reduces the difficulty of model training.

[0038] 5. The target enhancement algorithm for small sample data increases the amount of training data, improves the generalization ability of the model, better adapts to the environment and working conditions, and increases noise data to improve the robustness of the model.

[0039] 6. The parallel cascade deep learning method combines two parallel convolutional neural networks to achieve online preparation, classification and recognition of multiple types of foreign fibers. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 It is a schematic diagram of the structure of the system of the present invention.

[0042] Figure 2 Flow chart of the detection method of the present invention.

[0043] Figure 3 This is an image of foreign fibers captured by a color polarization camera.

[0044] Figure 4 It is the grayscale image after grayscale processing.

[0045] Figure 5 It is the working interface of the testing software in a specific instance.

[0046] Figure 6 It is the processing result of the present invention, wherein (a) is the foreign fiber image processed by the CLAHE algorithm (Contrast Limited Adaptive Histogram Equalization) in a specific example, and (b) is the result image of the original image captured by the polarization camera after being detected by the trained neural network.

[0047] Figure 7 It is the foreign fiber image extracted by the foreign fiber ROI (region of interest) algorithm in a specific example.

[0048] Figure 8 It is a flowchart of the transfer learning method for data imbalance.

[0049] Fig. 9 This is a processing flow chart of the fluorescence effect foreign fiber target detection model of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Example 1

[0052] like Figure 2 As shown, a cotton foreign fiber online detection method, the steps are as follows:

[0053] Step 1: The polarization camera collects images of the detection area of ​​the cotton irradiated by the laser in real time.

[0054] like Figure 2 As shown, the loosened and flattened cotton moves to the detection area along the conveyor belt; the line laser emitted by the 405nm blue-violet laser 4 is obliquely projected on the cotton in the detection area; the cotton in the detection area irradiated by the line laser is collected in real time by the color polarization camera 3, and the collected images are transmitted to the computer 7 in real time to obtain a data set composed of the collected images. The images collected in real time in the detection area are shown in FIG. Figure 3 As shown, Figure 3 The bright white lines are some heterogeneous fibers containing whitening agents, which will produce a "fluorescent effect" and produce color under ultraviolet light.

[0055] Foreign fibers generally have a dense structure. When the laser is projected vertically, the reflected light is too strong, creating a "glare"-like effect, which makes the contrast between foreign fibers and cotton weak. After changing the laser projection angle many times, it was found that when the angle was about 45°, the contrast was enhanced and the collected image had better contrast characteristics.

[0056] Step 2: The collected images form sample data, and the sample data is enhanced to obtain an enhanced sample set.

[0057] There are few labeled foreign fiber sample data, and the foreign fiber samples collected and produced by ourselves are limited. Data enhancement is performed on the small foreign fiber sample data. The data enhancement methods include at least two of the following: image mirroring, brightness change, Gaussian blur, pixel translation, size scaling, rotation transformation and / or pixel discarding.

[0058] Before training the sample data, seven target detection enhancement algorithms, including mirroring, brightness change, Gaussian blur, pixel translation, resizing, rotation transformation, and pixel discarding, are used for data enhancement. The left and right parts of the image are mirrored with the vertical axis of the image as the center; mirroring is the easiest image expansion to implement and has been proven to be effective for the data set. The brightness change is to multiply the grayscale value of each pixel by any value from 0.5 to 1.5. Illumination bias is one of the most common challenges in image recognition problems, so the effectiveness of brightness change is conceptually quite intuitive. The standard deviation of the Gaussian transformation of Gaussian blur is 0 to 3.0. The matrix of random values ​​sampled from the Gaussian distribution is added to the RGB pixels of the image. Adding noise to the image can help CNN (convolutional neural network) learn more powerful functions. Pixel translation is a translation of -20% to +20% (each axis). Moving image pixels can avoid positional deviations in the data. For example, in the face recognition data set, if all images are centered, using this data enhancement method can avoid errors caused by possible positional deviations. Resizing is to randomly scale the image to between 80% and 120%. Enlarging or reducing the image in a certain ratio does not change the content of the image, which can increase the generalization performance of the model. Rotation transformation is to randomly rotate the image to between -270° and +270°. Rotating the image can change the direction of the image content and increase the generalization ability of the network model. Pixel dropping is to randomly remove 1% to 10% of the pixels and set the grayscale values ​​of these pixels to 0. Pixel dropping can improve the performance of the model when the image is partially occluded. In addition, it can ensure that the network focuses on the entire image, not just part of it.

[0059] In the specific processing, the present invention performs seven image enhancement methods for each type of foreign fiber data collected, namely, mirroring, brightness, Gaussian, translation, scaling, rotation, and discarding. The number of expansions of each enhancement algorithm is the same as the original sample. For example, if there are 200 samples of foreign fiber in leaves collected, after seven enhancements, the total data set is 200+200*7=1600.

[0060] Step 3: Use transfer learning methods to pre-train the sample set to obtain pre-classified images.

[0061] To address the data imbalance problem, the present invention adopts a transfer learning method. Figure 8 As shown in FIG, the implementation method of the transfer learning method is as follows: for example, there are three foreign fiber samples P1, P2 and P3, the sample numbers of foreign fiber samples P1 and P3 are much smaller than that of foreign fiber sample P2, the sample data of foreign fiber samples P1 and P3 are combined into foreign fiber sample P0, foreign fiber samples P0 and P2 are passed into convolutional neural network I for training, and then the model weights of convolutional neural network I are transferred to convolutional neural network II for learning, and foreign fiber sample P0 is passed into convolutional neural network II for training to realize recognition and classification, as shown in FIG. Figure 8 As shown. Identification and classification results: Identification and classification of three foreign fiber samples P1, P2, and P3. This method is aimed at the situation of unbalanced sample ratio. Because there are many types of foreign fibers, some foreign fiber sample data sets are small, while others are large. Therefore, this transfer learning is used to input foreign fiber samples into our convolutional neural network training. The trained network model obtains the corresponding weight parameter information, and the network containing these weight parameters can be used to actually detect foreign fibers.

[0062] If the sample set to be collected for training is 10 kinds of foreign fibers, numbered 1 to 10, if the sample sizes of the three foreign fibers numbered 1, 3, and 5 are too small, and the sum of the sample sizes of the three is not much different from the remaining foreign fiber samples, then the 10-category recognition problem will be converted into an 8-category recognition problem, and the final recognition and classification result will still achieve the classification and positioning of these 10 foreign fibers.

[0063] There are many types of foreign fibers, and the imbalance of sample data leads to model non-convergence. The advantage of transfer learning is to balance the data ratio, prevent the model non-convergence problem caused by imbalanced ratio, change the three-classification task into two two-classification tasks, and reduce the difficulty of model training.

[0064] Step 4: The parallel cascade deep learning method is trained through the pre-classified images to obtain a training model. The training model recognizes and classifies the images collected by the polarization camera in step 1 to obtain the type and location of the foreign fibers.

[0065] Fluorescent substances containing whitening agents produce a fluorescent effect under 405nm ultraviolet light, and the corresponding grayscale image features are obvious. However, for foreign fibers with rich color features, grayscale images alone cannot be effectively classified. Therefore, a parallel cascade deep learning method is used for identification and classification.

[0066] The parallel cascade deep learning method includes an improved YOLOv5 algorithm and a fluorescence effect foreign fiber target detection algorithm. The improved YOLOv5 and the fluorescence effect foreign fiber target detection algorithm are cascaded in parallel. The improved YOLOv5 algorithm is used to identify foreign fibers with obvious classification color characteristics in the image. The fluorescence effect foreign fiber target detection algorithm is used to identify and classify foreign fibers that produce fluorescence effect under 405nm blue-violet laser irradiation.

[0067] In the final detection part, the image detected by the YOLOv5 algorithm is a color polarization camera. After the YOLOv5 algorithm is trained in transfer learning, it obtains a trained YOLOv5 model, which is used for actual detection. The data source for actual detection is the image collected in real time by the color polarization camera, and the collected color polarization image is input to the trained YOLOv5 model for detection. The image detected by the fluorescence effect foreign fiber target detection algorithm is the image after grayscale processing of the original image collected by the color polarization camera. After the fluorescence effect foreign fiber target detection algorithm is trained in transfer learning, it obtains a trained fluorescence effect foreign fiber target detection model, which is used for actual detection. The data source for actual detection is the image collected in real time by the color polarization camera, which is processed in one step, that is, grayscale processing converts the RGB image into a grayscale image, and the grayscale processed image is input to the trained fluorescence effect foreign fiber target detection model for detection.

[0068] The fluorescent effect foreign fiber target detection model includes a feature extraction part and a regression prediction part. The feature extraction part includes 6 sequentially connected convolutional layers. The convolution kernel size of each convolutional layer is 3*3, and the number is 16, 32, 64, 128, 256 and 512 respectively. The feature extraction part is used to extract the deep features of the pre-classified image; the regression prediction part includes an FCN structure, and adopts a feature combination method, that is, combining shallow features and deep features. The FCN structure includes 3 sequentially connected convolutional layers. The more convolutional layers, the better the feature extraction ability. The deep features are convoluted through 3 sequentially connected convolutional layers in the regression prediction part, and then upsampled. The upsampled features and shallow features are spliced ​​to generate fused features, and the fused features are used for regression prediction to obtain a prediction matrix.

[0069] The main feature of the grayscale image of foreign fibers with fluorescence effect is brightness information. In order to improve the detection speed, the processing method of the foreign fiber target detection model with fluorescence effect is as follows: the image after grayscale processing is extracted by the convolution layer of the feature extraction part to output the deep features. The deep features continue to be convolved in the regression prediction part. The convolution kernel size is 3*3, and the number of convolution kernels is 256, 128 and 64 respectively; then the deep features after convolution processing are upsampled to match the size of the shallow features, and then the sizes of different layers are spliced ​​to generate fused features; the fused features are used to continue regression prediction, and the input image is refined layer by layer in the feature extraction part to form deep abstract features with small size and large dimension. The deep features continue to be convolved in the regression prediction part. Since the feature extraction part is accompanied by downsampling, the detailed information of some features is lost. In the prediction regression stage, the deep features are upsampled to match the size of the shallow features, and then the sizes of different layers are spliced ​​to generate fused features, that is, feature combination, and the fused features are used to continue regression prediction, and finally a prediction matrix is ​​generated, such as Fig. 9 As shown, a 40*40*24 prediction matrix is ​​generated. The 40×40×24 prediction matrix divides the image into 40×40 grids, and each grid contains the confidence of the foreign fiber target and the coordinate information of the bounding box.

[0070] The grayscale processing of the present invention is aimed at foreign fibers that may produce fluorescence effects, because such foreign fibers will emit bright fluorescence after being irradiated with blue-violet light, so the brightness information becomes a significant distinguishing feature. The grayscale value of the grayscale image is equal to the brightness. The brightness and contrast in the color image are related, and the color image has more features than the grayscale image. The use of grayscale images can well distinguish foreign fibers that produce fluorescence effects, and to a certain extent improve the detection speed of the network model. Figure 4 As shown in FIG. 1 , the original image captured by the color polarization camera is converted to a grayscale image after being grayscaled. The grayscale image is input to the fluorescent effect foreign fiber target detection model for detection to obtain a processed image.

[0071] Transfer learning includes training, and transferring samples to the neural network is training. The YOLOv5 model and the fluorescent effect foreign fiber target detection model trained by the transfer learning method are cascaded in parallel, and actual detection can be carried out.

[0072] After the network is trained, the images collected in real time by the color polarization camera 3 are used for the final actual on-site detection. The camera will collect on-site images in real time, and then pass the collected images to the trained network model for detection, and output the final detection results. The input is input to the trained improved YOLOv5 algorithm for detection, and the type and location of foreign fibers are output in real time. The input end of the improved YOLOv5 algorithm processes the pre-classified images through Mosaic data enhancement. Mosaic data enhancement randomly crops four pictures and then splices them into one picture as training data, enriching the background of the picture.

[0073] The improved YOLOv5 algorithm uses an adaptive image scaling method to process images. Scaling is a preprocessing built into the YOLOv5 algorithm. Different images have different lengths and widths. The original images, i.e., the original images captured by the camera, are uniformly scaled to a standard size and then fed into the detection model. Adaptive anchor frame calculation is used. During network training, the improved YOLOv5 network in the YOLOv5 model outputs a predicted frame based on the initial anchor frame, and then compares it with the groundtruth, the correct label of the real frame. That is, by calculating the difference between the predicted frame and the real frame, the network parameters are updated in reverse and iterated.

[0074] The backbone network of the detection network is a Focus structure and a CSP structure connected in sequence. The original 640*640*3 image is input into the Focus structure, and the slicing operation is used to first convert it into a 320*320*12 feature map, and then a convolution operation of 32 convolution kernels is performed, and finally it becomes a 320*320*32 feature map. Reduce the number of floating-point operations performed per second and increase the speed. The CSP1_X structure in the CSP structure is applied to the backbone network Backbone, and the CSP2_X structure is applied to the network layer Neck, which solves the repeated gradient problem in other large convolutional network structures, reduces model parameters and the number of floating-point operations performed per second, which is of great significance to the YOLOv5 network, ensuring both the inference speed and accuracy and reducing the model size.

[0075] The network layer Neck of the detection network adopts the FPN+PAN structure; FPN+PAN draws on PANet from CVPR in 2018, which was mainly used in the field of image segmentation at that time, but Alexey (the author) split it and applied it to the Yolov4 network to further improve the feature extraction capability.

[0076] The improved Mish activation function used in the detection network simplifies the calculation formula when x is greater than 0, reduces the amount of calculation of the model, and improves the convergence speed of the model. It avoids the problem of gradient disappearance when x is less than 0. The improved function has non-monotonicity, which helps to maintain small negative values, thereby stabilizing the network gradient flow.

[0077] The output of the detection network includes a bounding box loss function and NMS non-maximum suppression.

[0078] The bounding box loss function adopts the CIoU Loss loss function, and:

[0079]

[0080] Among them, d = ρ (A, B) is the B frame and B gt The Euclidean distance of the coordinates of the center points of the boxes, and c is the diagonal distance of the smallest box that encloses them. α is the weight function, and v is used to measure the consistency of the aspect ratio:

[0081]

[0082]

[0083]

[0084] The final CIoU Loss is defined as:

[0085]

[0086] The CIoU Loss loss function avoids the problem that the gradient cannot be calculated in some cases when using IoU as the loss function. Based on GIoU, the penalty term of CIoU is based on the ratio of the distance between the center point and the diagonal distance, avoiding the problem of GIoU generating a large outer box when the distance between the two boxes is far, and the large loss value is difficult to optimize. Therefore, CIoULoss converges faster. The penalty term of CIoU adds an influencing factor a,v, through which the aspect ratio of the predicted box and the aspect ratio of the fitted target box can be taken into account.

[0087] The non-maximum suppression (NMS) adopts DIOU-NMS. Non-maximum suppression is to suppress elements that are not maximum values. It is used in target detection to extract target detection boxes with high confidence and suppress false detection boxes with low confidence. Effect: When calculating the loss function, DIOU needs to consider the position information of the center point of the bounding box, so some overlapping objects can also be regressed. Therefore, in the detection of overlapping targets, the effect of DIOU-NMS is better than that of traditional NMS. Using DIOU_nms, the detection effect of occlusion problems is improved.

[0088] The improved YOLOv5 algorithm has a depth coefficient depth_multiple=0.33 and a width coefficient width_multiple=0.33. The model volume is reduced by 53% and the detection speed is increased by 41%. There are a total of 4 versions of the YOLOv5 target detection network in the YOLOv5 model suitable for polarization detection systems, namely YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x. The YOLOv5s network is the network with the smallest network depth and width in the YOLOv5 series. The following three are continuously deepened and widened on this basis. The four structures control the depth and width of the network through the two parameters depth_multiple and width_multiple. Among them, depth_multiple controls the depth of the network, and width_multiple controls the width of the network. The deeper the depth of the network, the stronger the feature extraction capability of the network. The wider the width of the network, the stronger the learning ability of the network to extract features. The characteristic values ​​of the pictures collected by the polarization camera are less than those of the true color pictures. The deeper the depth and thicker the width of the network, the performance overflow may occur, and the corresponding training speed is slower, and the trained model is larger. Reducing the depth and width of the network is very helpful to improve the detection speed, and has little impact on the detection accuracy. YOLOv5s depth_multiple=0.33, width_multiple=0.50. The present invention has been tested many times on the basis of the YOLOv5s network. When depth_multiple is set to 0.33 and width_multiple is set to 0.33, the model volume is reduced by 53%, and the detection speed is increased by 41%, which meets the online detection requirements of our detection system.

[0089] Step 5: Perform histogram equalization on the identified and classified image through the CLAHE algorithm to obtain the foreign fiber position with the target detection frame.

[0090] The CLAHE algorithm enhances the details of polarized images and highlights foreign fiber targets. The specific implementation method is:

[0091] The size of the sliding window of the adaptive histogram equalization method is M×M, and the local mapping function is:

[0092]

[0093] Among them, CDF(i) is the cumulative distribution function of the local histogram of the sliding window, and M represents the size of the sliding window, for example, the sliding window size is 3*3. The derivative of the cumulative distribution function CDF(i) is the histogram Hist(i), so the slope S of the local mapping function m(i) is:

[0094]

[0095] For a histogram with a height greater than Hmax, the redundant part should be cut off, starting from the threshold T (rather than the maximum histogram height H max ) is used to truncate the histogram, and then the truncated part is evenly distributed over the entire grayscale range to ensure that the total histogram area remains unchanged, thereby increasing the entire histogram by a height L. Therefore, H max The following relationship is satisfied between H, T, and L: max =T+L.

[0096] Finally, the improved histogram values ​​are:

[0097]

[0098] By changing the maximum mapping function slope S max and the corresponding maximum histogram height H max , you can get images with different enhancement effects.

[0099] The implementation method of the CLAHE algorithm is as follows: the image after identification and classification is divided into non-overlapping sub-blocks of equal size, each sub-block contains M pixels, and the reference example is 8×8 blocks. The histogram of the sub-block is calculated; the pixels in the excess part of the histogram of the sub-block are reallocated by the clipping threshold clipLimit. The part exceeding the limit is clipped, and the clipped parts are evenly distributed to the other parts of the histogram, and then the grayscale values ​​of the pixels are reconstructed by histogram equalization, that is, bilinear interpolation is used to achieve reconstruction, so as to obtain an image after adaptive histogram equalization. After many experiments, the clipping threshold clipLimit=10 and the grid size of pixel equalization titleGridSize=(8,8) are taken. CLAHE can effectively suppress the enhancement of local contrast and noise amplification. clipLimit is the threshold of color contrast, and titleGridSize is the number of grids under which the histogram equalization operation is performed.

[0100] Step 6: The foreign fiber ROI algorithm extracts the foreign fiber position in the target detection frame and obtains the geometric parameters of the foreign fiber target.

[0101] The foreign fiber position output by the improved YOLOv5 algorithm and the fluorescent effect foreign fiber target detection algorithm is accompanied by a target detection frame. The detection frame roughly marks the foreign fiber position in the image. The foreign fiber ROI algorithm is used to extract the geometric parameters of the foreign fiber target object accurately, and the ground-truth of the region of interest ROI and the result of machine segmentation, that is, the trained target detection network, is obtained. When detecting an image, the position of the foreign fiber will be framed in the image, and that frame is the machine segmentation result.

[0102] Make comparisons and calculate evaluation indicators such as accuracy and recall.

[0103] Example 2

[0104] like Figure 1 As shown, a cotton foreign fiber online detection system includes a conveyor belt 2, the speed of the conveyor belt is 0-30m / min, the motor power is 60w, and the speed of the conveyor belt 2 is adjusted by operating the control button 6. The control button 6 is located on the operating table below the conveyor belt 2, the conveyor belt 2 is connected to the drive motor 1, the drive motor 1 and the control button 6 are both connected to the controller, and the controller controls the rotation of the drive motor 1 by issuing a command through the control button, thereby controlling the speed of the conveyor belt. The conveyor belt is located about 20cm directly below the color polarization camera and the 405nm blue-violet light laser. A bracket 5 is provided above the conveyor belt 2, and a polarization camera 3 and a laser 4 are installed on the bracket 5. The light emitting area of ​​the laser 4 is consistent with the image acquisition area of ​​the polarization camera 3, and the polarization camera 3 and the laser 4 are both connected to a computer 7. The computer 7 processes the image collected by the polarization camera 3 through an online detection method, extracts the foreign fiber target and outputs the geometric parameters of the foreign fiber target. It includes a blue-violet light laser with a wavelength of 405nm, a color polarization camera, a machine vision frame, a conveyor belt 2, a drive motor 1, a control button, and a computer; the polarization camera and the line laser are fixed on the machine vision frame, and the conveyor belt is located directly below the polarization camera and the line laser.

[0105] The laser 4 is a 405nm blue-violet laser with a power of 35mW and a divergence angle of 90°. Irradiation with a 405nm blue-violet laser can cause fluorescence effect in certain foreign fibers containing whitening agents. The light of the 405nm blue-violet laser is a polarized light source, and polarized light can well detect transparent foreign fibers in cotton. The sensor of the polarization camera 3 is equipped with a four-way pixel-level polarization filter of 0°, 45°, 90°, and 135°, with a resolution of 2448×2048 and a maximum frame rate of 74fps. The polarization filter can resist interference from stray light, has low requirements for ambient light during operation, and the collected images are more stable.

[0106] The polarization camera 3 and the laser 4 are fixed on the bracket 5 at a 45° angle in the horizontal direction, and the image collected by the 45° oblique projection has better contrast characteristics. The horizontal distance between the color polarization camera and the 405nm blue-violet laser is about 18 cm, and the vertical distance is about 1.5 cm.

[0107] The bracket 5 is provided with a height adjustment mechanism, and the polarization camera 3 and the laser 4 are both fixed on the height adjustment mechanism; the height adjustment mechanism includes a lifting rod with height adjustment, and the lifting rod is provided with a positioning column, and the positioning column matches the positioning hole on the bracket 5.

[0108] Depend on Figure 5 The detection interface displays the type and location of foreign fibers in real time. A detection interface was made using Python, and the results of the trained neural network detection were displayed on this detection interface. Figure 6 As shown in (a), the image after processing the detection of YOLOv5 and the fluorescence effect foreign fiber target detection model, the original image captured by the polarization camera after the trained neural network detection result is shown in the figure Figure 6 (b) shows the details of the foreign fiber target. Finally, the foreign fiber target is extracted by the foreign fiber ROI algorithm. Figure 7 As shown, it is processed Figure 6 (a) The actual geometric parameter information of the foreign fiber is obtained and output.

[0109] In the present invention, the loosened cotton is transmitted to the detection area composed of a polarization camera and a line laser via a conveyor belt. The image data collected in the detection area is processed by the improved YOLOv5 and fluorescent effect foreign fiber target detection algorithm in the computer 7 to output the type and location of the foreign fiber. After being processed by the CLAHE algorithm, the details of the foreign fiber target are enhanced. Finally, the foreign fiber target is extracted by the foreign fiber ROI algorithm and the geometric parameters of the foreign fiber are output. The present invention designs a transfer learning method for data imbalance and difficult model convergence, and proposes a parallel cascade deep learning method of YOLOv5 and fluorescent effect foreign fiber target detection algorithm, which finally realizes the identification and classification of foreign fibers, positioning and accurate measurement of geometric parameters.

[0110] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A cotton foreign fiber online detection method, characterized in that: The steps are as follows: Step 1: The polarization camera collects images of the detection area of ​​the cotton irradiated by the laser in real time; Step 2: The collected images form sample data, and data enhancement is performed on the sample data to obtain an enhanced sample set; Step 3: Use transfer learning method to pre-train the sample set to obtain pre-classified images; Step 4: The parallel cascade deep learning method is trained through the pre-classified images to obtain a training model. The training model recognizes and classifies the images collected by the polarization camera in step 1 to obtain the type and location of foreign fibers. The parallel cascade deep learning method comprises a parallel cascade of an improved YOLOv5 algorithm and a fluorescent effect foreign fiber target detection algorithm, wherein the improved YOLOv5 algorithm is used to identify foreign fibers with obvious classification color features in an image, and the fluorescent effect foreign fiber target detection algorithm is used to identify foreign fibers that produce a fluorescent effect under laser irradiation; the improved YOLOv5 algorithm is trained using pre-classified images to obtain a trained YOLOv5 model, and the fluorescent effect foreign fiber target detection algorithm is trained using the pre-classified images to obtain a fluorescent effect foreign fiber target detection model; in actual detection, the YOLOv5 model processes images collected in real time by a polarization camera, and the images detected by the fluorescent effect foreign fiber target detection model are images of the original images collected by the polarization camera after being grayscaled; Step 5: Perform histogram equalization on the image after identification and classification through the CLAHE algorithm to obtain the position of the foreign fiber with the target detection frame; Step 6: The foreign fiber ROI algorithm extracts the foreign fiber position in the target detection frame and obtains the geometric parameters of the foreign fiber target.

2. The cotton foreign fiber online detection method according to claim 1, characterized in that: The data enhancement method includes at least two of image mirroring, brightness change, Gaussian blur, pixel translation, size scaling, rotation transformation and / or pixel discarding.

3. The cotton foreign fiber online detection method according to claim 1 or 2, characterized in that: The fluorescent effect foreign fiber target detection algorithm includes a feature extraction part and a regression prediction part. The feature extraction part includes 6 sequentially connected convolutional layers for extracting deep features of the image; the regression prediction part includes an FCN structure, and the FCN structure includes 3 sequentially connected convolutional layers. The deep features and shallow features are combined in the regression prediction part through 3 sequentially connected convolutional layers for convolution processing, and then up-sampling is performed. The up-sampled features and shallow features are spliced ​​to generate fused features, and regression prediction is performed using the fused features to obtain a prediction matrix.

4. The cotton foreign fiber online detection method according to claim 3, characterized in that: The input end of the detection network of the improved YOLOv5 algorithm processes the pre-classified image through Mosaic data enhancement, the backbone network of the detection network is a Focus structure and a CSP structure connected in sequence, the detection network adopts an improved Mish activation function, and the network layer of the detection network adopts an FPN+PAN structure; the output end of the detection network includes a bounding box loss function and NMS non-maximum suppression; the depth coefficient depth_multiple of the improved YOLOv5 algorithm is 0.33, and the width coefficient width_multiple is 0.

33.

5. The cotton foreign fiber online detection method according to claim 4, characterized in that: The improved Mish activation function is: Where x is the independent variable, ω=4(x+1)+4e 2x +e 3x +e x (4x+6), and δ=2e x +e 2x +2.

6. The cotton foreign fiber online detection method according to claim 5, characterized in that: The bounding box loss function adopts the CIOU Loss loss function, and: Where d = ρ(A,B) is the distance between frame B and B gt The Euclidean distance of the coordinates of the center points of the boxes, and c is the diagonal distance of the smallest box that encloses them; α is the weight function, v is used to measure the consistency of the aspect ratio, and: The final CIoU Loss loss function is: The improved YOLOv5 algorithm uses an adaptive picture scaling method to process images and an adaptive anchor frame calculation, that is, by calculating the difference between the predicted frame and the real frame, the network parameters are updated in reverse and iterated; The NMS non-maximum suppression adopts DIOU-NMS.

7. The detection system of the cotton foreign fiber online detection method according to any one of claims 1, 2, 4-6, characterized in that: The invention comprises a conveyor belt (2), a bracket (5) is arranged above the conveyor belt (2), a polarization camera (3) and a laser (4) are mounted on the bracket (5), the light emitting area of ​​the laser (4) is consistent with the image acquisition area of ​​the polarization camera (3), and the polarization camera (3) and the laser (4) are both connected to a computer (7), and the computer (7) processes the image acquired by the polarization camera (3) through an online detection method, extracts foreign fiber targets and outputs the geometric parameters of the foreign fiber targets.

8. The cotton foreign fiber online detection system according to claim 7, characterized in that: The laser (4) is a 405nm blue-violet laser, and the sensor of the polarization camera (3) is equipped with four-way pixel-level polarization filters of 0°, 45°, 90°, and 135°; the bracket (5) is provided with a height adjustment mechanism, and the polarization camera (3) and the laser (4) are both fixed on the height adjustment mechanism; the height adjustment mechanism includes a lifting rod, and a positioning column is provided on the lifting rod, and the positioning column matches the positioning hole on the bracket (5).

9. The cotton foreign fiber online detection system according to claim 8, characterized in that: The polarization camera (3) and the laser (4) are fixed on a bracket (5) at an angle of 45° in the horizontal direction.

Images