A method, system, electronic device, and medium for classifying sewing threads.

Abstract

This invention discloses a method, system, electronic device, and medium for classifying stitching. The method includes: acquiring stitching images to be identified when the production line is running and the stitching packaging is obscured; inputting the stitching images to be identified into a pre-trained stitching classification model to obtain stitching classification results; wherein the training process of the stitching classification model includes: acquiring stitching images and preprocessing the acquired stitching images; extracting features from the preprocessed stitching images and inputting the features corresponding to the stitching images into a random forest classifier to obtain pre-classification results; using the stitching images and the corresponding pre-classification results as training datasets, extracting samples from the training datasets using the Margin algorithm, and training the stitching classification model using the extracted samples.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial image recognition classification, and in particular to a sewing line classification method and system, an electronic device and a medium. BACKGROUND

[0002] In the PS packaging machine sewing image recognition project of a chemical enterprise, the entire packaging process of the production line is mainly as follows: plastic particles are placed in a woven bag by an injection molding machine, and after being filled, the woven bag is sealed by a sewing machine, and the sealing part is adhered by using a wrapping paper. However, due to the influence of complex factors such as production technology and production environment, various quality problems will inevitably occur in the sewing process of the sewing line, and some severely flawed bags will leak particles on the production line, thereby causing the production line to be temporarily suspended and requiring manual processing before resuming operation. In addition, some packaging bags with slight flaws still have the problem of particle leakage and poor packaging sealing quality after being sold to customers, thereby causing certain losses to the factory. Therefore, an automatic detection program is needed to identify the flaws of the packaging bag, so as to perform a push bag action and alarm processing on the packaging bag with severe flaws, thereby improving the production efficiency of the factory and the packaging quality of the packaging bag.

[0003] Through related camera collection equipment, 1w+ sewing line packaging bag images can be collected every day, so the collected data is very complete. However, in the project scenario, the image data samples of the sewing line with flaws only account for about 1%. After manual classification and labeling, the number of each flaw category is uneven, and the proportion of a certain type of flaw is nearly 40%, while the proportion of several types is only 2%-4%.

[0004] The training of the traditional deep learning image classification model requires a large amount of image data, and requires that the data of each category is relatively balanced, so as to train a high-quality image classification model with high accuracy, recall rate, etc. Although a large amount of data samples are generated every day, the sample data category distribution is extremely unbalanced, and the proportion of flaw samples is extremely small, so it is necessary to collect as many flaw samples as possible, and the collection of flaw samples is a long-term process. According to statistics, a field expert needs about 5 hours to label a day's samples, and since the samples need to be collected for a long time, the cost of manual labeling is huge.

[0005] Traditional active learning can significantly reduce the labeling cost of manual labeling while reducing the classification performance to a certain extent. However, due to the extremely unbalanced distribution of real samples and the condition limitation of sample sampling, the traditional active learning scheme has the following two problems:

[0006] (1) Unable to accurately control the category distribution of the training set sample data;

[0007] (2) Unable to collect every day appearance defect sample completely, extended sample collection time.

[0008] Therefore, it is urgent to propose a sewing thread classification method. SUMMARY

[0009] In view of the deficiencies in the prior art, the application discloses a sewing thread classification method, system, electronic device and medium.

[0010] The first aspect of the embodiment of the application provides a sewing thread classification method, which comprises the following steps:

[0011] When the pipeline is running and the sewing thread package is in a shielding state, a sewing thread image to be identified is collected;

[0012] The sewing thread image to be identified is input into a pre-trained sewing thread classification model to obtain a sewing thread classification result; wherein the training process of the sewing thread classification model comprises the following steps: collecting a sewing thread image, and pre-processing the collected sewing thread image; performing feature extraction on the pre-processed sewing thread image, inputting the features corresponding to the sewing thread image into a random forest classifier to obtain a pre-classification result; taking the sewing thread image and the corresponding pre-classification result as a training data set, extracting samples by using a Margin algorithm, and training the sewing thread classification model by using the extracted samples.

[0013] The second aspect of the embodiment of the application provides a sewing thread classification system, which comprises the following steps:

[0014] A sewing thread image collection module is configured to collect a sewing thread image to be identified when the pipeline is running and the sewing thread package is in a shielding state;

[0015] A classification result acquisition module is configured to input the sewing thread image to be identified into a pre-trained sewing thread classification model to obtain a sewing thread classification result; wherein the training process of the sewing thread classification model comprises the following steps: collecting a sewing thread image, and pre-processing the collected sewing thread image; performing feature extraction on the pre-processed sewing thread image, inputting the features corresponding to the sewing thread image into a random forest classifier to obtain a pre-classification result; taking the sewing thread image and the corresponding pre-classification result as a training data set, extracting samples from the training data set by using a Margin algorithm, and training the sewing thread classification model by using the extracted samples.

[0016] A classification result processing module is configured to output an alarm signal and / or a push package signal according to the defect type corresponding to the sewing thread classification result; comprising: if the sewing thread classification result is a first grade, no processing is required; if the sewing thread classification result is a second grade or a third grade, an alarm signal is output; and if the sewing thread classification result is a third grade, a push package signal is output.

[0017] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, the memory being coupled to the processor; wherein the memory is configured to store program data, and the processor is configured to execute the program data to implement the above-mentioned sewing thread classification method.

[0018] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the above-mentioned sewing thread classification method.

[0019] Compared with the prior art, the present application has the following beneficial effects:

[0020] 1. In the training process of the sewing thread classification model, the present application firstly uses a random forest classifier to pre-classify the collected sewing thread images, so as to solve the problem of extremely small proportion of defect samples and data imbalance. At the same time, the present application reduces the manual labeling cost and improves the data collection efficiency.

[0021] 2. In the training process of the sewing thread classification model, the present application divides the data into two stages according to the number of data. In the first stage, the second and third grade samples are directly added to the training data set; the first grade samples are sampled by the Margin algorithm for uncertainty and then added to the training data set; in the second stage, the first, second and third grade samples are sampled by the Margin algorithm for uncertainty as the training data set; through the active learning method, the problems of extremely unbalanced data samples in the real environment, high manual labeling cost and the need for a large number of training samples for the deep learning model are solved. DETAILED DESCRIPTION

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0023] Figure 1 is a schematic diagram of the sewing thread classification method provided by the present application;

[0024] Figure 2 is a schematic diagram of the sewing thread classification device provided by the present application;

[0025] Figure 3 is a flowchart of collecting sewing thread images provided by the present application;

[0026] Figure 4 is a schematic diagram of the sewing thread image classification standard provided by the present application;

[0027] Figure 5 is a flow chart of feature extraction of an image processing algorithm provided by an embodiment of the present application;

[0028] Figure 6 is a comparison chart of edge detection operators provided by an embodiment of the present application;

[0029] Figure 7 is a schematic diagram of a union set 5x5 connection kernel provided by an embodiment of the present application;

[0030] Figure 8 is a schematic diagram of three-region mean value calculation provided by an embodiment of the present application;

[0031] Figure 9 is a schematic diagram of sewing line edge feature region mean value calculation provided by an embodiment of the present application;

[0032] Figure 10 is a schematic diagram of edge paper edge feature region mean value calculation provided by an embodiment of the present application;

[0033] Figure 11 is an effect diagram of sewing line image pre-classification provided by an embodiment of the present application;

[0034] Figure 12 is a schematic diagram of training a sewing line classification model provided by an embodiment of the present application;

[0035] Figure 13 is a processing flow chart of an alarm signal provided by an embodiment of the present application;

[0036] Figure 14 is a processing flow chart of a push package signal provided by an embodiment of the present application;

[0037] Figure 15 is a schematic diagram of a sewing line classification system provided by an embodiment of the present application;

[0038] Figure 16 is a schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0039] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0040] It should be noted that the features in the following embodiments and implementation manners can be combined with each other without conflict.

[0041] AsFigure 1 As shown in the accompanying drawings, the application discloses a sewing thread classification method, which is realized based on a sewing thread classification device. Figure 2 As shown in the accompanying drawings, the device comprises an industrial camera, an iologik, an infrared detector, a light supplement lamp, a pipeline central control console and a server cabinet. The industrial camera is directly connected to the server cabinet through a network card port to provide continuous high-definition video information. The iologik is an Ethernet remote I / O device, which is also directly connected to the server cabinet through a network card port. The server cabinet can query the port information of the iologik or output information at the DO port of the iologik. The infrared detector is connected to the DI port of the iologik. When the sewing thread packaging bag blocks the infrared detector, the infrared detector outputs a signal at the DI port, so that the server cabinet obtains the blocking information by querying the iologik. The light supplement lamp is used to enhance the features of the sewing thread, which is beneficial to model training and model discrimination. The pipeline central control console inputs the pipeline start-stop signal to the DI port of the iologik, and the server cabinet obtains the running state of the pipeline by querying the iologik. The server is used to connect the iologik and the industrial camera, and run the sewing thread recognition software to obtain the state information and video signal of the hardware device, so as to realize automatic sewing thread image acquisition.

[0042] The method specifically comprises the following steps:

[0043] Step S1, when the pipeline is running and the sewing thread packaging is in a blocking state, a sewing thread image to be identified is collected.

[0044] Specifically, as shown in the accompanying drawings, Figure 3 The step S1 comprises the following sub-steps:

[0045] Step S101, a frame of high-definition image information is obtained by the industrial camera.

[0046] Step S102, the running state of the pipeline is queried. If the running state of the pipeline is paused, a frame of high-definition image information is re-collected in step S101. Otherwise, step S103 is entered.

[0047] Step S103, the infrared blocking state is obtained. If there is no blocking, a frame of high-definition image information is re-collected in step S101. Otherwise, step S104 is entered.

[0048] Step S104, it is judged whether it is the first time to enter the blocking state, that is, whether a new sewing thread packaging appears. If yes, the frame number counter is reset to 0. Otherwise, the frame number counter is accumulated.

[0049] Step S105, judging whether the frame number counter reaches the specified key frame, that is, whether the sewing thread packaging bag runs to the optimal collection position, if not, returning to step S101 to collect a frame of high-definition image information again, if yes, saving the collected sewing thread image.

[0050] Step S2, inputting the sewing thread image to be identified into the pre-trained sewing thread classification model to obtain a sewing thread classification result.

[0051] The sewing thread classification model adopts an Efficient-Net model structure. The Efficient-Net model is an Efficient-Net_V2_M model with pre-trained weights.

[0052] The training process of the sewing thread classification model includes:

[0053] Step S201, collecting a sewing thread image and pre-processing the collected sewing thread image; performing feature extraction on the pre-processed sewing thread image, inputting the features corresponding to the sewing thread image into a random forest classifier to obtain a pre-classification result.

[0054] It should be noted that the image processing classification algorithm of step S201 is a heuristic algorithm based on the characteristics of the sewing thread image. Compared with the deep learning model, the algorithm has less requirement for the training set and has a certain classification effect. The sewing thread image is pre-classified into the first level (i.e., A level), the second level (i.e., B level) and the third level (i.e., C level), which can greatly reduce the cost of manual annotation in the early stage of defect sample collection. The sewing thread image and its classification standard are shown in Figure 4 , wherein the first level is a normal sewing thread; the second level is a sewing thread with appearance defects, including: no edge strip, incomplete edge strip, and wrinkled edge strip; and the third level is a sewing thread with material leakage defects, including: no sewing thread, uneven feeding of the bag to the sewing thread, and sewing the sewing thread to the outside of the wrinkle.

[0055] As shown in Figure 5 , the step S201 specifically includes the following steps:

[0056] Step S20101, extracting the features of the sewing thread image by using a canny operator to obtain a canny edge feature map.

[0057] Further, the canny operator is used for the sewing thread image, wherein the canny_threshold1 is 30 and the canny_threshold2 is 90, so that the canny edge feature map can be extracted. The comparison chart of various edge detection operators is shown in Figure 6As shown in the figure, the edge features extracted by the Canny operator are binary, fine and continuous, and thus are used as the edge feature extraction operator.

[0058] Step S20102: Perform erosion and expansion operations on the Canny edge feature map.

[0059] Further, perform erosion operation on the Canny edge feature map using a 2x1 convolution kernel. Since the vertical edge features are 1 pixel wide, the vertical edge features can be removed, so as to highlight the feature information of the top and bottom edges of the seam line and the edge paper. Then, perform 4 expansion operations on the map using a 2x1 convolution kernel. Since the erosion operation weakens the connection of the horizontal edge features, the expansion operation can connect and strengthen the horizontal edge features. The specific effect can be seen in the Canny map in Figure 5 It can be observed that the horizontal edge features are clear and continuous.

[0060] Step S20103: Traverse the Canny edge feature map. When a white pixel is traversed, use the 5x5 connection kernel to connect the adjacent white pixels to the same connected block by taking the current pixel as the center, so as to segment each continuous horizontal edge feature.

[0061] Suppose that a white pixel is currently traversed, which is placed in the center (3, 3) position of the convolution kernel. The surrounding white pixels are assumed as shown in Figure 7 By using the 5x5 connection kernel, it can be considered that the white pixels at positions (1, 5), (2, 1), (5, 1) and (5, 4) are in the same connected block as the white pixel at the current (3, 3) position. By this operation, each continuous horizontal edge feature can be segmented, and its total pixel number, width and other information can be obtained.

[0062] Step S20104: In the horizontal edge feature, there are still some noise feature information. Therefore, traverse the Canny edge feature map. When a white pixel is traversed, query the size of the connected block to which it belongs by using the find operation of the union set, and if the size is less than 100, it is considered to belong to interference noise, and the white pixel at this point is set to black, otherwise no processing is performed.

[0063] Step S20105: Traverse the Canny edge feature map obtained from step S20104 in order. When a white pixel is traversed, set the point as (x, y), then return to the corresponding (x, y) position in the original map, and use the 3-region mean calculation method as shown in Figure 8The method places (x, y) at the center of a 9x29 pixel block and divides the pixel block into three regions of 9x10, 9x9, and 9x10 from top to bottom, and then calculates the average gray values avg2, avg1, and avg3 of the three regions on the original image;

[0064] When the white pixel (x, y) in the canny image is an edge feature of the sewing line, as shown in FIG. 6A, it can be known that the average gray values of the three regions must satisfy the following condition: Figure 9

[0065] avg1<avg2 and avg1<avg3

[0066] When the white pixel (x, y) in the canny image is an edge feature of the edge paper, as shown in FIG. 6B, it can be known that the average gray values of the three regions must satisfy the following condition: Figure 10

[0067] avg2<avg1<avg3

[0068] According to the above judgment conditions, the features of the sewing line and the edge paper can be extracted from the canny image, as shown in the third and fifth rows of the sewing line extraction image and the edge paper extraction image in FIG. 6C. Then, through the union-find set algorithm, the white pixels belonging to the same connected block of the horizontal edge features of the sewing line and the edge paper are queried, and the white pixels in the same connected block are filled, so that the fourth and sixth rows of the sewing line completion image and the edge paper completion image in FIG. 6D are obtained. Figure 5 Figure 5

[0069] Step S20106: According to the calculation of the sewing line completion image and the edge paper completion image, by traversing all the white pixels in the image, the width, height, position, pixel point number, and relative position information of the sewing line and the edge paper can be calculated.

[0070] Step S20107: By calculating the feature information of all the sewing line images in the training set and adding the corresponding label value, the best parameters of the random forest are obtained using the random forest Bayesian optimization parameter tuning method, and the random forest model is trained. Then, for a new sewing line image, the calculated feature information is input into the random forest classifier to obtain a pre-classification result.

[0071] According to the above algorithm flow steps, the algorithm has a certain classification effect on a small sample data set with a total sample number of 1482, and the effect diagram is shown in FIG. 7. The recall rate for B and C class defects is good. Figure 11

[0072] ​​​​​Step S202, using the sewing thread image and the corresponding pre-classification result as the training data set, using the Margin algorithm to extract samples from the training data set, and using the extracted samples to train the sewing thread classification model.

[0073] As shown in Figure 12 , the step S202 is divided into first stage and second stage, the difference between the first stage and the second stage is the richness degree of the number of defect samples and the classification effect of the deep learning model, generally considered that the number of each defect sample is less than 5k or the classification effect of the deep learning model is weaker than the image processing classification algorithm as the first stage, otherwise as the second stage.

[0074] Step S20201, the first stage, the unlabeled sample pool is periodically supplemented by the image acquisition module, then using the image processing classification algorithm, the unlabeled sample is pre-classified as A, B, C level. For B, C level defect samples, due to the shortage of defect samples in the early stage, all of them need to be supplemented to the training set, so they are directly added to the labeled sample pool after expert labeling. For A level, it is a large proportion of normal samples, and the characteristics are extremely similar, mainly through the sample acquisition strategy of active learning, using the Margin algorithm in uncertainty sampling. That is, first predict a batch of images through the Efficient-net classification model, set and respectively the two largest results of the sample x predicted, subtracting and sorting, that is, the closer the difference between the first two results, the more difficult it is to judge this example, the specific formula is as follows:

[0075]

[0076] Then the extracted samples are labeled by experts and added to the labeled sample pool. The updated labeled sample pool can be used to train a new Efficient-net classification model, and the random forest model in the image processing classification algorithm can also be updated;

[0077] S20202, the second stage, the unlabeled sample pool is still periodically supplemented by the image acquisition module, then using the image processing classification algorithm, the unlabeled sample is pre-classified as A, B, C level. Through the sample acquisition strategy of active learning, using the Margin algorithm in uncertainty sampling, sample sampling is carried out for A, B, C level respectively, and the sampling number of each level can be set according to the demand, because the pre-classification function of the image processing classification algorithm, when the sampling number of each level is set, the proportion of each category can be well controlled, realizing the construction of balanced labeled data set. Then update the Efficient-net classification model and the random forest model in the image processing classification algorithm using the labeled sample pool.

[0078] Step S3, after obtaining the prediction result of the sewing thread image, an alarm and a pushing action are performed according to the defect category.

[0079] The alarm action has a specific process as shown in Figure 13 , which includes the following steps:

[0080] Step S301, obtaining the judgment result of the current sewing thread image from the model judgment module;

[0081] Step S302, judging the defect category, and if the A level is not processed, the process is ended, and if the B and C levels are entered into S3;

[0082] Step S303, judging whether the current state is in an alarm state, if yes, the alarm time is extended by 4S, otherwise, the current state is set as an alarm state, and the alarm time is set as 4S, and after the expiration, the non-alarm state is automatically entered.

[0083] The pushing action display diagram is shown in Figure 14 , which needs to add a pushing infrared 2 before the reverse packing area, and the average time of the packaging bag passing through the camera infrared 1 and the pushing infrared 2 is measured, so that the pushing delay time T1 is obtained. The pushing duration T2 is manually set by the worker, and in the actual scene, it is 3S. When the packaging bag is detected by the camera infrared 1, and the defect category is C level after the image acquisition module and the model judgment module, the pushing action needs to be performed. Since the pushing area is a distance away from the detection area, and the speed of the assembly line is constant, the pushing delay time T1 can be measured to know when the packaging bag enters the pushing area. When the assembly line is paused, the pushing delay time also needs to be extended, that is, T1+T3. After entering the pushing area, the reverse packing machine of the assembly line control console is controlled by the server cabinet, so that the transmission direction of the assembly line is changed, and the automatic screening of the defective packaging bag is realized.

[0084] As shown in Figure 15 , the embodiment of the application further provides a sewing thread classification system, which includes:

[0085] A sewing thread image acquisition module is used to acquire the sewing thread image to be identified when the assembly line is running and the sewing thread packaging is in a shielding state.

[0086] The classification result acquisition module is configured to input the to-be-identified sewing thread image into a pre-trained sewing thread classification model to obtain a sewing thread classification result. The training process of the sewing thread classification model includes: collecting sewing thread images and pre-processing the collected sewing thread images; performing feature extraction on the pre-processed sewing thread images, inputting the features corresponding to the sewing thread images into a random forest classifier to obtain a pre-classification result; taking the sewing thread images and the corresponding pre-classification results as a training data set, extracting samples from the training data set by using a Margin algorithm, and training the sewing thread classification model by using the extracted samples.

[0087] The classification result processing module is configured to output an alarm signal and / or a push package signal according to the defect type corresponding to the sewing thread classification result. The classification result processing module includes: if the sewing thread classification result is a first grade, no processing is needed; if the sewing thread classification result is a second grade or a third grade, an alarm signal is output; and if the sewing thread classification result is the third grade, a push package signal is output.

[0088] As to the system in the above-described embodiments, the specific manners in which the various modules perform operations have been described in detail in the embodiments of the method, and thus will not be described in detail here.

[0089] As to the system embodiments, since they basically correspond to the method embodiments, the related parts can be referred to the part of the method embodiments. The above-described system embodiments are merely illustrative, and the units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, that is, they can be located in one place or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purposes of the application scheme according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0090] Correspondingly, the application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the sewing thread classification method as described above. As shown in the Figure 16 The sewing thread classification method provided by the embodiment of the application is a hardware structure diagram of any device with data processing capability. In addition to the processor, the memory and the network interface shown in the Figure 16 In addition to the processor, the memory and the network interface shown in the

[0091] Correspondingly, the application further provides a computer readable storage medium, which stores computer instructions, and the instructions are executed by a processor to realize the sewing thread classification method. The computer readable storage medium can be an internal storage unit of any device with data processing capability, such as a hard disk or a memory. The computer readable storage medium can also be an external storage device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit of any device with data processing capability and the external storage device. The computer readable storage medium is used to store the computer program and other programs and data required by the device with data processing capability, and can also be used to temporarily store data that has been output or will be output.

[0092] The specific embodiments are only an explanation of the application, and are not a limitation of the application. Those skilled in the art can make modifications to the embodiments without creative contribution, as long as the modifications are within the scope of the claims of the application.

Claims

1. A method of classifying a sewing thread, characterized by, The method comprises: When the sewing thread package is in the shielding state during the pipeline operation, a sewing thread image to be identified is collected; The sewing thread image to be identified is input into a pre-trained sewing thread classification model to obtain a sewing thread classification result; wherein the training process of the sewing thread classification model comprises: collecting a sewing thread image and pre-processing the collected sewing thread image; performing feature extraction on the pre-processed sewing thread image, inputting the features corresponding to the sewing thread image into a random forest classifier to obtain a pre-classification result; taking the sewing thread image and the corresponding pre-classification result as a training data set, using a Margin algorithm to extract samples, and training the sewing thread classification model using the extracted samples; The pre-classification result comprises: a first grade, a second grade, and a third grade; the first grade is a normal sewing thread; the second grade is a sewing thread with appearance defects, including: no edge strip, incomplete edge strip, and wrinkled edge strip; and the third grade is a sewing thread with material leakage defects, including: no sewing thread, uneven bag feeding, and sewing thread sewn to the outside of the wrinkle; Taking the sewing thread image and the corresponding pre-classification result as a training data set, using a Margin algorithm to extract samples, and training the sewing thread classification model using the extracted samples comprises: In a first stage, the sewing thread image is collected and input into a pre-trained random forest classifier to obtain a pre-classification result; the second grade and third grade samples are directly added to the training data set; the first grade samples are sampled for uncertainty using the Margin algorithm and then added to the training data set; and the training data set is used to train the sewing thread classification model; In a second stage, the sewing thread image is collected and input into a pre-trained random forest classifier to obtain a pre-classification result; the first grade, second grade, and third grade samples are sampled for uncertainty using the Margin algorithm as a training data set; and the training data set is used to train the sewing thread classification model.

2. The method of claim 1, wherein, When the sewing thread package is in the shielding state for the first time during the pipeline operation, the sewing thread image to be identified is collected, comprising: Configuring a frame number counter and setting a key frame; Collecting an image when the sewing thread package is in the shielding state during the pipeline operation; Determining whether it is the first shielding state; if it is not the first shielding state, resetting the frame number counter to 0; if it is the first shielding state, accumulating the frame number counter; Determining whether the frame number counter reaches the key frame; if it does not reach the key frame, re-collecting an image; if it reaches the key frame, taking the collected image as the sewing thread image to be identified.

3. The method of claim 1, wherein: Collecting a sewing thread image and pre-processing the collected sewing thread image; Performing feature extraction on the pre-processed sewing thread image, inputting the features corresponding to the sewing thread image into a random forest classifier to obtain a pre-classification result, comprising: Using a canny operator to extract features of the sewing thread image to obtain a canny edge feature map; Performing erosion and expansion operations on the canny edge feature map; Traverse the Canny edge feature map, when traversing to a white pixel, through the union set algorithm, take the current pixel as the center, and merge the adjacent white pixels into the same connected block to segment out each horizontal edge feature; Traverse the Canny edge feature map, when traversing to a white pixel, query the size of the connected block to which it belongs, and when the size of the connected block is less than the threshold, the white pixel is interference noise, and the white pixel is set to black; Traverse the Canny edge feature map, when traversing to a white pixel, then go back to the position corresponding to the white pixel in the original sewing line image, divide the white pixel into upper, middle and lower three regions in the original sewing line image, and calculate the average gray value of each region; According to the average gray value of the upper, middle and lower three regions, the sewing line edge feature and the edge feature of the edge paper are extracted from the Canny edge feature map; Based on the sewing line edge feature and the edge feature of the edge paper, through the union set algorithm, the connected block to which the white pixels of the sewing line edge feature and the edge paper horizontal edge feature belong is queried, and the white pixels in the same connected block are filled to obtain the sewing line completion map and the edge paper completion map; Traverse the white pixels in the sewing line completion map and the edge paper completion map to obtain the width, height, position, pixel number and relative position information of the sewing line and the edge paper, which are used as the sewing line image feature information; Input the sewing line image feature information into the trained random forest classifier to obtain a pre-classification result.

4. A method of classifying sewing thread according to claim 3, characterized in that According to the average gray value of the upper, middle and lower three regions, the sewing line edge feature and the edge feature of the edge paper are extracted from the Canny edge feature map, including: The average gray value of the upper region is recorded as the second average gray value, the average gray value of the middle region is recorded as the first average gray value, and the average gray value of the lower region is recorded as the third average gray value; When the white pixel in the Canny edge feature map is the edge feature of the sewing line, the average gray values of the three regions need to satisfy: the first average gray value < the second average gray value, and the first average gray value < the third average gray value; When the white pixel in the Canny edge feature map is the edge feature of the edge paper, the average gray values of the three regions need to satisfy: the second average gray value < the first average gray value < the third average gray value.

5. A method of classifying sewing thread as defined in claim 1, wherein The sewing line classification method further includes: According to the defect type corresponding to the sewing line classification result, output an alarm signal and / or a push package signal; including: if the sewing line classification result is the first grade, no processing is needed; if the sewing line classification result is the second grade or the third grade, an alarm signal is output; if the sewing line classification result is the third grade, a push package signal is output.

6. A system for classifying a sewn line, the system comprising: The system is used to realize the sewing line classification method of any one of claims 1-5, and the system includes: A sewing line image acquisition module is used to acquire a sewing line image to be identified when a pipeline is running and a sewing line package is in a shielding state; The classification result acquisition module is configured to input the to-be-identified sewing thread image into a pre-trained sewing thread classification model to obtain a sewing thread classification result. The training process of the sewing thread classification model includes: collecting sewing thread images and pre-processing the collected sewing thread images; performing feature extraction on the pre-processed sewing thread images, inputting the features corresponding to the sewing thread images into a random forest classifier to obtain a pre-classification result; taking the sewing thread images and the corresponding pre-classification results as a training data set, extracting samples from the training data set by using a Margin algorithm, and training the sewing thread classification model by using the extracted samples. The classification result processing module is configured to output an alarm signal and / or a push package signal according to the defect type corresponding to the sewing thread classification result. The classification result processing module includes: if the sewing thread classification result is a first grade, no processing is needed; if the sewing thread classification result is a second grade or a third grade, an alarm signal is output; and if the sewing thread classification result is the third grade, a push package signal is output.

7. An electronic device comprising a memory and a processor, characterized in that The memory is coupled to the processor, and is configured to store program data. The processor is configured to execute the program data to implement the sewing thread classification method according to any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the sewing thread classification method according to any one of claims 1-5.

Images