Image recognition-based double-twisted yarn breakage detection method, system, device and medium

Abstract

The application relates to a kind of image recognition-based double-twist yarn breakage detection method, system, equipment and medium, and its technical scheme points are: obtaining target image;Identify whether the target image contains a pulley, the pulley is the pulley for conveying yarn on double-twist machine, in the case where the target image is identified to contain pulley, the foreground area containing pulley is cut out to obtain cutting image, the arrow direction of pulley on target image is identified and classified to obtain arrow direction category, the corresponding feature map is obtained by HOG feature extraction on cutting image, whether the pulley and reference pulley are the same pulley is judged by KCF algorithm on feature map, if yes, then the arrow direction category is recorded in the reference direction category set corresponding to reference pulley, if not, then the state of reference pulley is judged according to reference direction category set, whether yarn breakage is judged according to the state of reference pulley, the application has the effect of low identification difficulty and higher detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of yarn breakage detection technology, and more specifically, to a method, system, device, and medium for detecting double-twist yarn breakage based on image recognition. Background Technology

[0002] During the twisting process of yarn, yarn may break during the twisting process using a twisting machine. In a real factory environment, the inspection of broken yarn usually requires continuous inspection to detect whether broken yarn occurs during the twisting process, and then untie the broken yarn.

[0003] The prior art discloses a document with publication number CN106093052A entitled "A Method for Detecting Broken Yarn," which includes: controlling a pan-tilt unit to move along a guide rail, with a CCD camera mounted on the pan-tilt unit, the CCD camera capturing video signals in real time during the movement along the guide rail; extracting frames from the video signals at preset intervals, and detecting whether there is a broken yarn based on the extracted frames; if a broken yarn is detected based on the extracted frames, extracting image content of a specified area from the frames containing the broken yarn, and setting a sign in the specified area; matching and recognizing the identification characters represented by the sign in the specified area using a preset image template, and sending the recognized identification characters to a monitoring device. However, this method for detecting broken yarn has the following problems: in actual production, the yarn thickness varies, its uses are different, and the processing technology is also different, making it difficult to directly capture whether the yarn is broken using an industrial camera. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a method, system, device, and medium for detecting double-twist yarn breakage based on image recognition, which has the advantages of low recognition difficulty and higher detection accuracy.

[0005] The above-mentioned technical objective of the present invention is achieved through the following technical solution:

[0006] A method for detecting yarn breakage based on image recognition, comprising:

[0007] S100, Acquire the target image;

[0008] S200. Identify whether the target image contains a pulley, wherein the pulley is a pulley for conveying yarn on a twisting machine. If so, proceed to step S300.

[0009] S300: Cut out the foreground region containing the pulley to obtain the corresponding cut image, perform HOG feature extraction on the cut image to obtain the corresponding feature map, identify the arrow direction of the pulley in the target image and classify it to obtain the corresponding arrow direction category;

[0010] S400. Perform the KCF algorithm on the feature map to determine whether the pulley and the reference pulley are the same pulley. If yes, proceed to step S500; otherwise, proceed to step S600.

[0011] S500: Record the arrow pointing category in the reference pointing category set corresponding to the reference pulley, and return to step S100.

[0012] S600. Determine the state of the reference pulley according to the reference pointing category set, and determine whether the yarn is broken according to the state of the reference pulley.

[0013] Optionally, after step S300, the method further includes:

[0014] Determine whether the cut image is the first cut image containing a pulley; if not, proceed to step S400.

[0015] If so, the pulley is used as the reference pulley, the arrow pointing category is used as the reference pointing category set corresponding to the reference pulley, and the process returns to step S100.

[0016] Optionally, the step of extracting HOG features from the cut image to obtain the corresponding feature map includes:

[0017] The cut image is converted to grayscale, and the color space of the grayscale cut image is normalized using gamma correction to obtain a normalized image.

[0018] Calculate the gradient of each pixel in the normalized image;

[0019] The normalized image is divided into multiple blocks, and each block is further divided into multiple cell units;

[0020] The gradient histograms of each cell unit are statistically analyzed, and the gradient histograms of all cell units within each block are concatenated to obtain the HOG feature map corresponding to each block.

[0021] The corresponding feature map is obtained by concatenating the HOG feature maps of all the blocks.

[0022] Optionally, step S400 includes:

[0023] The feature map is input into the trained KCF filter to obtain the response map and response peak value;

[0024] If the peak response value is greater than a preset multiple of the average historical peak response value, it is determined that the pulley and the reference pulley are the same pulley, and step S500 is executed. If the peak response value is not greater than a preset multiple of the average historical peak response value, it is determined that the pulley and the reference pulley are not the same pulley, and step S600 is executed.

[0025] Optionally, the training method for the KCF filter includes:

[0026] Obtain base samples, wherein the base samples include: a feature map of a reference pulley;

[0027] Map the base samples to a high-dimensional space;

[0028] The first sample set is obtained by performing a cyclic shift on the base samples mapped to the high-dimensional space.

[0029] The filter is trained and updated using the first sample set. The filter coefficients and the first sample set are updated using linear interpolation.

[0030] Optionally, step S200 includes:

[0031] The target image is standardized and then input into the trained YOLO model to obtain the detection result;

[0032] If the detection result indicates that the target image contains a pulley, the target image includes a foreground region containing a pulley and a background region not containing a pulley, and step S300 is executed;

[0033] If the detection result indicates that the target image does not contain a pulley, then return to step S100.

[0034] Optionally, the training method for the YOLO model includes:

[0035] Obtain a second sample set, wherein the second sample set includes: multiple target images containing pulleys and multiple target images not containing pulleys;

[0036] The pulleys in the samples of the second sample set are labeled to obtain the true labeled images;

[0037] The YOLO model is trained using the second sample set, and the output of the YOLO model is compared with the real labeled images to obtain the training error.

[0038] The training error is backpropagated in the YOLO model to update the parameters of the YOLO model. After multiple iterations of training, a trained YOLO model is obtained.

[0039] Optionally, after step S600, the method further includes:

[0040] The pulley is used as the next reference pulley, the reference pointing category set is changed to the historical pointing category set, and the arrow pointing category is used as the reference pointing category set corresponding to the next reference pulley.

[0041] A twisted yarn breakage detection system based on image recognition includes:

[0042] The image acquisition module is used to acquire the target image;

[0043] An image recognition module is used to identify whether the target image contains a pulley, wherein the pulley is a pulley for conveying yarn on a twisting machine; if so, the image cutting module is activated.

[0044] The cutting and extraction module is used to cut out the foreground area containing the pulley to obtain the corresponding cut image, perform HOG feature extraction on the cut image to obtain the corresponding feature map, identify the arrow direction of the pulley in the target image and classify it to obtain the corresponding arrow direction category;

[0045] The pulley determination module is used to use the KCF algorithm to determine whether the pulley and the reference pulley are the same pulley. If they are, the pointing record module is started; if not, the determination update module is started.

[0046] The pointing recording module is used to record the arrow pointing category in the reference pointing category set corresponding to the reference pulley, and then return to the startup image acquisition module;

[0047] The yarn breakage determination module is used to determine the state of the reference pulley based on the reference pointing category set, and to determine whether the yarn is broken based on the state of the reference pulley.

[0048] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0049] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described above.

[0050] In summary, the present invention has the following beneficial effects: it identifies the arrow direction of the pulley on the cutting area and classifies it to obtain the arrow direction category; it uses the KCF algorithm on the feature map to determine whether the pulley on the target image and the reference pulley are the same pulley; if they are the same pulley, the arrow direction category of the reference pulley is recorded in the reference direction category set corresponding to the reference pulley; until the pulley in the acquired target image and the reference pulley are not the same pulley, the state of the reference pulley is determined according to the reference direction category set; if there are multiple different arrow direction categories in the reference direction category set, it is determined that the reference pulley is in a rotating state and the yarn conveyed by the reference pulley is not broken; if the arrow direction categories in the reference direction category set are the same, it is determined that the reference pulley is in a stationary state and the yarn conveyed by the reference pulley is broken, thus realizing the detection of broken yarn. Since the yarn is relatively thin, the detection accuracy of this application is higher than that of directly detecting whether the yarn is broken. Attached Figure Description

[0051] Figure 1 This is a flowchart illustrating an embodiment of the method provided by the present invention;

[0052] Figure 2 This is a flowchart illustrating another embodiment of the method provided by the present invention;

[0053] Figure 3 This is a structural block diagram of the image recognition-based double-twist yarn breakage detection system provided by the present invention;

[0054] Figure 4 This is an internal structural diagram of the computer device in an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Several embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein.

[0056] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances. The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature.

[0057] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0058] This invention provides a method for detecting yarn breakage based on image recognition, such as... Figure 1 As shown, it includes:

[0059] S100. Acquire the target image; wherein the target image is the current frame image acquired by the mobile camera;

[0060] S200. Identify whether the target image contains a pulley, wherein the pulley is a pulley for conveying yarn on a twisting machine. If yes, proceed to step S300; otherwise, return to step S100.

[0061] S300: Cut out the foreground region containing the pulley to obtain a cut image, extract HOG features from the cut image to obtain the corresponding feature map, identify the arrow direction of the pulley on the target image and classify it to obtain the arrow direction category;

[0062] S400. Perform the KCF algorithm on the feature map to determine whether the pulley and the reference pulley are the same pulley. If yes, proceed to step S500; otherwise, proceed to step S600.

[0063] S500: Record the arrow pointing category in the reference pointing category set corresponding to the reference pulley, and return to execute step S100;

[0064] S600: Identify the state of the reference pulley according to the reference pointing category set, and determine whether the yarn is broken according to the state of the reference pulley.

[0065] In practical applications, a twisting machine is equipped with a rotating pulley for conveying yarn. When the yarn is intact, the pulley rotates; when the yarn breaks, the pulley remains stationary. The pulley is equipped with an arrow label or a rubber sleeve with an arrowhead. As the pulley rotates, the direction of the arrow changes. In this application, the arrow direction is divided into eight categories: "up," "upper right," "right," "lower right," "lower," "lower left," "left," and "upper left." This is achieved through the AGV (Automated Guided Vehicle). The automated guided vehicle (AGV) robot is equipped with a camera, forming a moving camera. The moving camera moves along a predetermined route, capturing images during its movement. The current frame image from the moving camera is used as the target image. The target image is then identified. If a pulley is detected in the target image, the region containing the pulley is cut out, resulting in a cut-out region. The arrow pointing to the pulley in the cut-out region is identified and classified into arrow pointing categories. Feature extraction is performed on the cut-out image to obtain corresponding feature maps. Then, the KCF algorithm is applied to the feature maps to determine whether the pulley in the target image is the same as a reference pulley (the pulley previously captured by the moving camera), i.e., whether the moving camera has moved to another pulley. The system takes a picture of another pulley at the same location. If the pulley is the same as the previous one, the arrow pointing category of the pulley is recorded in the reference pointing category set corresponding to the reference pulley. This process continues until the pulley in the target image is not the same as the previously photographed pulley (i.e., the reference pulley). The state of the reference pulley is then determined based on the reference pointing category set. If there are multiple different arrow pointing categories in the reference pointing category set, it is determined that the reference pulley is rotating and the yarn being transported by the reference pulley is not broken. If the arrow pointing categories in the reference pointing category set are the same, it is determined that the reference pulley is stationary and the yarn being transported by the reference pulley is broken. This achieves the detection of broken yarn. Since the yarn is relatively thin, compared to directly detecting whether the yarn is broken, this application has lower recognition difficulty and higher detection accuracy.

[0066] Furthermore, after step S300, the method further includes:

[0067] Determine whether the cut image is the first cut image containing a pulley; if not, proceed to step S400.

[0068] If so, the pulley is used as a reference pulley, the arrow pointing category is recorded to form a reference pointing category set, and the process returns to step S100.

[0069] In practical applications, during the shooting process of a mobile camera, the first few frames captured may not contain a pulley. In this case, it is first determined whether the cut image is the first cut image containing a pulley, that is, whether the target image is the first target image containing a pulley, or whether the pulley currently captured is the first pulley captured by the mobile camera. If so, the pulley is used as the reference pulley, and the arrow pointing category is used as the reference pointing category set corresponding to the reference pulley. Then, the process returns to step S100. If not, step S400 is executed.

[0070] Further, the step of extracting HOG features from the cut image to obtain a feature map includes:

[0071] The cut image is converted to grayscale, and the color space of the grayscale cut image is normalized using gamma correction to obtain a normalized image.

[0072] Calculate the gradient of each pixel in the normalized image;

[0073] The normalized image is divided into multiple blocks, and each block is further divided into multiple cell units;

[0074] The gradient histograms of each cell unit are statistically analyzed, and the gradient histograms of all cell units within each block are concatenated to obtain the HOG feature map corresponding to each block.

[0075] The corresponding feature map is obtained by concatenating the HOG feature maps of all the blocks.

[0076] Specifically, the cut image is grayscaled to reduce interference from color information. Gamma correction is then used to normalize the color space of the grayscaled cut image, thereby adjusting the image contrast and reducing the impact of local shadows and lighting changes, thus minimizing lighting interference. The gradient of each pixel in the normalized image is calculated. In this application, the gradient of pixel I(x,y) in the normalized image is dx(x,y)=[I(x+1,y)-I(x-1,y)] / 2; dy(x,y)=[I(i,x+1)-I(x,y-1)] / 2; where dx(x,y) represents the horizontal gradient of pixel I(x,y), and dy(x,y) represents the vertical gradient of pixel I(x,y). The gradient magnitude and gradient direction of pixel I(x,y) are respectively... This allows for better capture of contour information and further reduces the interference of illumination. The gradient histograms of each cell unit are statistically analyzed. For example, a histogram of 9 bins is used to statistically analyze the gradient information of each cell unit. That is, each pixel in the cell unit is weighted and projected onto the histogram using the gradient direction (mapped to a fixed angle range) to obtain the gradient histogram corresponding to each cell unit. The gradient histograms of all cell units in each block are concatenated to obtain the HOG feature map corresponding to each block. The HOG feature maps of all blocks are concatenated to obtain the corresponding feature map.

[0077] Further, step S400 includes:

[0078] The feature map is input into the trained KCF filter to obtain the response map and response peak value;

[0079] If the peak response value is greater than a preset multiple of the average historical peak response value, it is determined that the pulley and the reference pulley are the same pulley, and step S600 is executed. If the peak response value is not greater than a preset multiple of the average historical peak response value, it is determined that the pulley and the reference pulley are not the same pulley, and step S700 is executed.

[0080] In practical applications, after the feature map is input into the trained KCF filter, the KCF filter performs filtering calculations on the feature map to obtain the output response map and response peak. If the cut image corresponding to the pulley is not the first cut image containing the pulley, at least one target image of the current reference pulley is usually obtained before obtaining the target image corresponding to the pulley. The historical response peak is obtained by the KCF filter filtering calculation on the feature map corresponding to the current reference pulley. The response peak is compared with a preset multiple of the average historical response peak. If the response peak is greater than the preset multiple of the average historical response peak, it means that the reference pulley has not been lost and the pulley and the reference pulley are the same pulley. The response peak corresponding to the pulley is taken as the historical response peak corresponding to the current reference pulley. If the response peak is not greater than the preset multiple of the average historical response peak, it means that the reference pulley has been lost. The camera is moved to the next pulley. If the pulley and the reference pulley are not the same pulley, the response peak corresponding to the pulley is taken as the historical response peak corresponding to the next reference pulley.

[0081] Furthermore, the training method for the KCF filter includes:

[0082] Obtain base samples, wherein the base samples include: a feature map of a reference pulley;

[0083] Map the base samples to a high-dimensional space;

[0084] The first sample set is obtained by performing a cyclic shift on the base samples mapped to the high-dimensional space.

[0085] The filter is trained and updated using the first sample set. The filter coefficients and the first sample set are updated using linear interpolation.

[0086] In practical applications, the base samples are mapped to a high-dimensional space. The mapped base samples are then cyclically shifted to obtain positive and negative samples, which constitute the first sample set. These positive and negative samples include the base samples. The positive and negative samples form a cyclic matrix, which is used to train the filter. The filter coefficients are then updated using linear interpolation, specifically by setting an update step size β. The update formula is shown below:

[0087] α=(1-β)α′+βnew_α

[0088] x=(1-β)x′+βnew_x

[0089] Where α represents the filter coefficients and x represents the first sample set consisting of positive and negative samples; by inputting the base samples into the filter, that is, by inputting the feature map corresponding to the reference pulley into the filter, the response peak corresponding to the reference pulley can be obtained, thereby obtaining the corresponding historical response peak.

[0090] Further, step S200 includes:

[0091] The target image is standardized and then input into the trained YOLO model to obtain the detection result;

[0092] If the detection result indicates that the target image contains a pulley, the target image includes a foreground region containing a pulley and a background region not containing a pulley, and step S300 is executed;

[0093] If the detection result indicates that the target image does not contain a pulley, then return to step S100.

[0094] Specifically, the target image is standardized by scaling it to 448*448 pixels to conform to the input standard of the YOLO model. The standardized target image is then input into the trained YOLO model to obtain detection results. These results are either the target image contains a pulley or it does not. If the detection result indicates the target image contains a pulley, the foreground region containing the pulley and the background region not containing the pulley are obtained. Then, step S300 is executed, which involves cutting out the foreground region containing the pulley to obtain a cut image. If the detection result indicates the target image does not contain a pulley, the process returns to step S100 to continue acquiring the target image.

[0095] Furthermore, the training method for the YOLO model includes:

[0096] Obtain a second sample set, wherein the second sample set includes: multiple target images containing pulleys and multiple target images not containing pulleys;

[0097] The pulleys in the samples of the second sample set are labeled to obtain the true labeled images;

[0098] The YOLO model is trained using the second sample set, and the output of the YOLO model is compared with the real labeled images to obtain the training error.

[0099] The training error is backpropagated in the YOLO model to update the parameters of the YOLO model. After multiple iterations of training, a trained YOLO model is obtained.

[0100] In practical applications, the YOLO model mainly includes multiple convolutional layers, two fully connected layers, and an output layer. The output layer uses a linear function as the activation function. After inputting samples from the second sample set into the YOLO model, the output result is obtained. The output result is compared with the real labeled image to obtain the training error. In this application, in addition to labeling the pulleys in the samples of the second sample set, the category pointed to by the arrows on the pulleys can also be labeled, so that the trained YOLO model can also recognize the category pointed to by the arrows on the pulleys. The YOLO model combines recognition and localization into a single network design, which makes it fast in processing. Moreover, this unified network design also allows training and prediction to be performed end-to-end, which is very simple. Since the entire image is put into the network for prediction, the background prediction error rate is low.

[0101] Furthermore, after step S600, the following steps are also included:

[0102] The pulley is used as the next reference pulley, the reference pointing category set is changed to the historical pointing category set, and the arrow pointing category is used as the reference pointing category set corresponding to the next reference pulley.

[0103] In practical applications, when the pulley in the acquired target image is not the same pulley as the previously captured pulley (i.e., the reference pulley), the method further includes: using the pulley as the next reference pulley, changing the reference pointing category set to the historical pointing category set, and using the arrow pointing category as the next reference pointing category set, so as to identify whether the yarn being transported by the next reference pulley is broken.

[0104] The present invention provides an image recognition-based method for detecting yarn breakage. It identifies the arrow direction of pulleys on the cutting area and classifies them into arrow direction categories. The KCF algorithm is used on the feature map to determine whether the pulley in the target image is the same as a reference pulley. If they are the same pulley, the arrow direction category of the reference pulley is recorded in the corresponding reference direction category set. This process continues until the pulley in the acquired target image is not the same as the reference pulley. Then, the state of the reference pulley is determined based on the reference direction category set. If there are multiple different arrow direction categories in the reference direction category set, it is determined that the reference pulley is rotating and the yarn being transported by the reference pulley is not broken. If the arrow direction categories in the reference direction category set are the same, it is determined that the reference pulley is stationary and the yarn being transported by the reference pulley is broken. This method achieves yarn breakage detection. Because the yarn is relatively thin, the detection accuracy of this application is higher than that of directly detecting yarn breakage.

[0105] like Figure 2 As shown, the present invention also provides an image recognition-based double-twist yarn breakage detection system, comprising:

[0106] Image acquisition module 10 is used to acquire target images;

[0107] The image recognition module 20 is used to identify whether the target image contains a pulley, wherein the pulley is a pulley for conveying yarn on a twisting machine; if so, the image cutting module is activated.

[0108] The cutting and extraction module 30 is used to cut out the foreground area containing the pulley to obtain the corresponding cut image, perform HOG feature extraction on the cut image to obtain the corresponding feature map, identify the arrow direction of the pulley in the target image and classify it to obtain the corresponding arrow direction category;

[0109] The pulley judgment module 40 is used to use the KCF algorithm to judge whether the pulley and the reference pulley are the same pulley. If they are, the pointing record module is started; if not, the judgment update module is started.

[0110] The pointing recording module 50 is used to record the arrow pointing category in the reference pointing category set corresponding to the reference pulley, and then return to the startup image acquisition module;

[0111] The yarn breakage determination module 60 is used to determine the state of the reference pulley based on the reference pointing category set, and to determine whether the yarn is broken based on the state of the reference pulley.

[0112] Specific limitations regarding the image recognition-based double-twist yarn breakage detection system can be found in the limitations of the image recognition-based double-twist yarn breakage detection method described above, and will not be repeated here. Each module of the aforementioned image recognition-based double-twist yarn breakage detection system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0113] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an image recognition-based method for detecting yarn breakage.

[0114] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0115] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to perform the following steps:

[0116] S100, Acquire the target image;

[0117] S200. Identify whether the target image contains a pulley, wherein the pulley is a pulley for conveying yarn on a twisting machine. If so, proceed to step S300.

[0118] S300: Cut out the foreground region containing the pulley to obtain the corresponding cut image, perform HOG feature extraction on the cut image to obtain the corresponding feature map, identify the arrow direction of the pulley in the target image and classify it to obtain the corresponding arrow direction category;

[0119] S400. Perform the KCF algorithm on the feature map to determine whether the pulley and the reference pulley are the same pulley. If yes, proceed to step S500; otherwise, proceed to step S600.

[0120] S500: Record the arrow pointing category in the reference pointing category set corresponding to the reference pulley, and return to step S100.

[0121] S600. Determine the state of the reference pulley according to the reference pointing category set, and determine whether the yarn is broken according to the state of the reference pulley.

[0122] In one embodiment, after step S300, the method further includes:

[0123] Determine whether the cut image is the first cut image containing a pulley; if not, proceed to step S400.

[0124] If so, the pulley is used as the reference pulley, the arrow pointing category is used as the reference pointing category set corresponding to the reference pulley, and the process returns to step S100.

[0125] In one embodiment, the step of extracting HOG features from the cut image to obtain the corresponding feature map includes:

[0126] The cut image is converted to grayscale, and the color space of the grayscale cut image is normalized using gamma correction to obtain a normalized image.

[0127] Calculate the gradient of each pixel in the normalized image;

[0128] The normalized image is divided into multiple blocks, and each block is further divided into multiple cell units;

[0129] The gradient histograms of each cell unit are statistically analyzed, and the gradient histograms of all cell units within each block are concatenated to obtain the HOG feature map corresponding to each block.

[0130] The corresponding feature map is obtained by concatenating the HOG feature maps of all the blocks.

[0131] In one embodiment, step S400 includes:

[0132] The feature map is input into the trained KCF filter to obtain the response map and response peak value;

[0133] If the peak response value is greater than a preset multiple of the average historical peak response value, it is determined that the pulley and the reference pulley are the same pulley, and step S600 is executed. If the peak response value is not greater than a preset multiple of the average historical peak response value, it is determined that the pulley and the reference pulley are not the same pulley, and step S700 is executed.

[0134] In one embodiment, the training method for the KCF filter includes:

[0135] Obtain base samples, wherein the base samples include: a feature map of a reference pulley;

[0136] Map the base samples to a high-dimensional space;

[0137] The first sample set is obtained by performing a cyclic shift on the base samples mapped to the high-dimensional space.

[0138] The filter is trained and updated using the first sample set. The filter coefficients and the first sample set are updated using linear interpolation.

[0139] In one embodiment, step S200 includes:

[0140] The target image is standardized and then input into the trained YOLO model to obtain the detection result;

[0141] If the detection result indicates that the target image contains a pulley, the target image includes a foreground region containing a pulley and a background region not containing a pulley, and step S300 is executed;

[0142] If the detection result indicates that the target image does not contain a pulley, then return to step S100.

[0143] In one embodiment, the training method of the YOLO model includes:

[0144] Obtain a second sample set, wherein the second sample set includes: multiple target images containing pulleys and multiple target images not containing pulleys;

[0145] The pulleys in the samples of the second sample set are labeled to obtain the true labeled images;

[0146] The YOLO model is trained using the second sample set, and the output of the YOLO model is compared with the real labeled images to obtain the training error.

[0147] The training error is backpropagated in the YOLO model to update the parameters of the YOLO model. After multiple iterations of training, a trained YOLO model is obtained.

[0148] In one embodiment, after step S600, the method further includes:

[0149] The pulley is used as the next reference pulley, the reference pointing category set is changed to the historical pointing category set, and the arrow pointing category is used as the reference pointing category set corresponding to the next reference pulley.

[0150] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0151] S100, Acquire the target image;

[0152] S200. Identify whether the target image contains a pulley, wherein the pulley is a pulley for conveying yarn on a twisting machine. If so, proceed to step S300.

[0153] S300: Cut out the foreground region containing the pulley to obtain the corresponding cut image, perform HOG feature extraction on the cut image to obtain the corresponding feature map, identify the arrow direction of the pulley in the target image and classify it to obtain the corresponding arrow direction category;

[0154] S400. Perform the KCF algorithm on the feature map to determine whether the pulley and the reference pulley are the same pulley. If yes, proceed to step S500; otherwise, proceed to step S600.

[0155] S500: Record the arrow pointing category in the reference pointing category set corresponding to the reference pulley, and return to step S100.

[0156] S600. Determine the state of the reference pulley according to the reference pointing category set, and determine whether the yarn is broken according to the state of the reference pulley.

[0157] In one embodiment, after step S300, the method further includes:

[0158] Determine whether the cut image is the first cut image containing a pulley; if not, proceed to step S400.

[0159] If so, the pulley is used as the reference pulley, the arrow pointing category is used as the reference pointing category set corresponding to the reference pulley, and the process returns to step S100.

[0160] In one embodiment, the step of extracting HOG features from the cut image to obtain the corresponding feature map includes:

[0161] The cut image is converted to grayscale, and the color space of the grayscale cut image is normalized using gamma correction to obtain a normalized image.

[0162] Calculate the gradient of each pixel in the normalized image;

[0163] The normalized image is divided into multiple blocks, and each block is further divided into multiple cell units;

[0164] The gradient histograms of each cell unit are statistically analyzed, and the gradient histograms of all cell units within each block are concatenated to obtain the HOG feature map corresponding to each block.

[0165] The corresponding feature map is obtained by concatenating the HOG feature maps of all the blocks.

[0166] In one embodiment, step S400 includes:

[0167] The feature map is input into the trained KCF filter to obtain the response map and response peak value;

[0168] If the peak response value is greater than a preset multiple of the average historical peak response value, it is determined that the pulley and the reference pulley are the same pulley, and step S600 is executed. If the peak response value is not greater than a preset multiple of the average historical peak response value, it is determined that the pulley and the reference pulley are not the same pulley, and step S700 is executed.

[0169] In one embodiment, the training method for the KCF filter includes:

[0170] Obtain base samples, wherein the base samples include: a feature map of a reference pulley;

[0171] Map the base samples to a high-dimensional space;

[0172] The first sample set is obtained by performing a cyclic shift on the base samples mapped to the high-dimensional space.

[0173] The filter is trained and updated using the first sample set. The filter coefficients and the first sample set are updated using linear interpolation.

[0174] In one embodiment, step S200 includes:

[0175] The target image is standardized and then input into the trained YOLO model to obtain the detection result;

[0176] If the detection result indicates that the target image contains a pulley, the target image includes a foreground region containing a pulley and a background region not containing a pulley, and step S300 is executed;

[0177] If the detection result indicates that the target image does not contain a pulley, then return to step S100.

[0178] In one embodiment, the training method of the YOLO model includes:

[0179] Obtain a second sample set, wherein the second sample set includes: multiple target images containing pulleys and multiple target images not containing pulleys;

[0180] The pulleys in the samples of the second sample set are labeled to obtain the true labeled images;

[0181] The YOLO model is trained using the second sample set, and the output of the YOLO model is compared with the real labeled images to obtain the training error.

[0182] The training error is backpropagated in the YOLO model to update the parameters of the YOLO model. After multiple iterations of training, a trained YOLO model is obtained.

[0183] In one embodiment, after step S600, the method further includes:

[0184] The pulley is used as the next reference pulley, the reference pointing category set is changed to the historical pointing category set, and the arrow pointing category is used as the reference pointing category set corresponding to the next reference pulley.

[0185] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0186] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A method for detecting yarn breakage based on image recognition, characterized in that, include: S100, Acquire the target image; S200. Identify whether the target image contains a pulley, wherein the pulley is a pulley for conveying yarn on a twisting machine. If so, proceed to step S300. S300. Cut out the foreground area containing the pulley to obtain the corresponding cut image. Perform HOG feature extraction on the cut image to obtain the corresponding feature map. Identify the arrow label or the arrow direction of the rubber sleeve with arrow on the pulley in the target image and classify it to obtain the corresponding arrow direction category. The arrow direction category is used to characterize the rotation direction of the pulley. Determine whether the cut image is the first cut image containing a pulley. If yes, use the current pulley as the reference pulley, use the current arrow pointing category as the reference pointing category set corresponding to the reference pulley, and return to step S100. If no, proceed to step S400. S400, The reference pulley is a pulley determined based on the prior target image; the KCF algorithm is used to determine whether the pulley and the reference pulley are the same pulley. If yes, then step S500 is executed; if no, then step S600 is executed. S500: Record the arrow pointing category in the reference pointing category set corresponding to the reference pulley, and return to step S100. S600. Determine the state of the reference pulley according to the reference pointing category set, specifically: if the reference pointing category set contains multiple different arrow pointing categories, then determine that the reference pulley is in a rotating state; if all the arrow pointing categories in the reference pointing category set are the same, then determine that the reference pulley is in a stationary state, and then determine whether the yarn is broken according to the state of the reference pulley, wherein the rotating state indicates that the yarn is not broken, and the stationary state indicates that the yarn is broken.

2. The method for detecting yarn breakage based on image recognition according to claim 1, characterized in that, The step of extracting HOG features from the cut image to obtain the corresponding feature map includes: The cut image is converted to grayscale, and the color space of the grayscale cut image is normalized using gamma correction to obtain a normalized image. Calculate the gradient of each pixel in the normalized image; The normalized image is divided into multiple blocks, and each block is further divided into multiple cell units; The gradient histograms of each cell unit are statistically analyzed, and the gradient histograms of all cell units within each block are concatenated to obtain the HOG feature map corresponding to each block. The corresponding feature map is obtained by concatenating the HOG feature maps of all the blocks.

3. The method for detecting yarn breakage based on image recognition according to claim 1, characterized in that, Step S400 includes: The feature map is input into the trained KCF filter to obtain the response map and response peak value; If the peak response value is greater than a preset multiple of the average historical peak response value, it is determined that the pulley and the reference pulley are the same pulley, and step S500 is executed. If the peak response value is not greater than a preset multiple of the average historical peak response value, it is determined that the pulley and the reference pulley are not the same pulley, and step S600 is executed.

4. The image recognition-based yarn breakage detection method according to claim 3, characterized in that, The training method for the KCF filter includes: Obtain base samples, wherein the base samples include: a feature map of a reference pulley; Map the base samples to a high-dimensional space; The first sample set is obtained by performing a cyclic shift on the base samples mapped to the high-dimensional space. The filter is trained and updated using the first sample set. The filter coefficients and the first sample set are updated using linear interpolation.

5. The method for detecting yarn breakage based on image recognition according to claim 1, characterized in that, Step S200 includes: The target image is standardized and then input into the trained YOLO model to obtain the detection result; If the detection result indicates that the target image contains a pulley, the target image includes a foreground region containing a pulley and a background region not containing a pulley, and step S300 is executed; If the detection result indicates that the target image does not contain a pulley, then return to step S100.

6. The method for detecting yarn breakage based on image recognition according to claim 5, characterized in that, The training methods for the YOLO model include: Obtain a second sample set, wherein the second sample set includes: multiple target images containing pulleys and multiple target images not containing pulleys; The pulleys in the samples of the second sample set are labeled to obtain the true labeled images; The YOLO model is trained using the second sample set, and the output of the YOLO model is compared with the real labeled images to obtain the training error. The training error is backpropagated in the YOLO model to update the parameters of the YOLO model. After multiple iterations of training, a trained YOLO model is obtained.

7. The method for detecting yarn breakage based on image recognition according to claim 1, characterized in that, Following step S600, the method further includes: The pulley is used as the next reference pulley, the reference pointing category set is changed to the historical pointing category set, and the arrow pointing category is used as the reference pointing category set corresponding to the next reference pulley.

8. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images