Yarn-out state detection method, detection device, electronic apparatus and storage medium

Abstract

To provide a yarn-out state detection method, a device, an apparatus and a storage medium, in which these relate to the field of computer, in particular to the fields of detection technology, artificial intelligence technology and neural network model technology.SOLUTION: The method includes: collecting a first image and laser reflection data of a target yarn path in a spinning box; obtaining an image feature of the target yarn path according to the first image; obtaining a laser feature of the target yarn path according to the laser reflection data; and using a yarn-out state detection model to obtain a yarn-out state of the target yarn path according to the image feature and the laser feature. The use of the technical solution of the disclosed embodiment can effectively improve the accuracy and efficiency of the yarn-out state detection and improve the stability of the entire spinning process and the product quality.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to the field of computers, and in particular, to a method, apparatus, device, and storage medium for detecting the state of yarn extrusion.

Background Art

[0002] In the chemical fiber industry, the speed of extruding yarn from a melt spinning container is very fast, and the fiber yarn extruded from the spinneret is extremely thin and visually appears as a strip-shaped ghost. In the prior art, it is difficult to accurately confirm the state of the yarn visually. After abnormal situations such as yarn breakage or yarn floating occur, or blade cleaning is performed on the spinneret at regular intervals, the quality of yarn extrusion is guaranteed.

Summary of the Invention

Problems to be Solved by the Invention

[0003] The present disclosure provides a method, apparatus, device, and storage medium for detecting the state of yarn extrusion to solve or alleviate one or more technical problems in the prior art.

Means for Solving the Problems

[0004] According to a first aspect, the present disclosure provides a method for detecting the state of yarn extrusion, comprising: collecting a first image and laser reflection data of a target yarn path in a spinning container; obtaining image features of the target yarn path based on the first image; obtaining laser features of the target yarn path based on the laser reflection data; and obtaining the yarn extrusion state of the target yarn path based on the image features and the laser features by using a yarn extrusion state detection model.

[0005] According to a second aspect, the present disclosure provides a device for detecting the state of yarn extrusion, comprising: A collection module for collecting a first image of a target yarn path and laser reflection data in a spinning container, An image feature determination module for obtaining image features of the target yarn path based on the first image, A laser feature determination module for obtaining laser features of the target yarn path based on the laser reflection data, A detection module for obtaining the yarn extrusion state of the target yarn path based on the image features and the laser features using a yarn extrusion state detection model, and a detection device for the yarn extrusion state is provided.

[0006] The third aspect is to obtain training image features of the target yarn path based on a sample image of the target yarn path, To obtain training laser features of the target yarn path based on laser sample data of the target yarn path, Using a first detection model to obtain a prediction result of the yarn extrusion state of the target yarn path based on the training image features and the training laser features, Based on the prediction result and the true result of the yarn extrusion state of the target yarn path, determining a loss function, Based on the loss function, updating the parameters of the first detection model to obtain a trained yarn extrusion state detection model, and a training method for the yarn extrusion state detection model is provided. The fourth aspect is an image feature extraction module for obtaining training image features of the target yarn path based on a sample image of the target yarn path, A laser feature extraction module for obtaining training laser features of the target yarn path based on laser sample data of the target yarn path, A prediction module for obtaining a prediction result of the yarn extrusion state of the target yarn path based on the training image features and the training laser features using a first detection model, A loss function determination module for determining a loss function based on the prediction result and the true result of the yarn extrusion state of the target yarn path, A training module is provided for updating the parameters of the first detection model based on a loss function to obtain a trained yarn extrusion state detection model, and a training apparatus for a yarn extrusion state detection model including the training module is provided.

[0007] The fifth aspect is at least one processor, and a memory communicably connected to the at least one processor, and includes instructions executable by the at least one processor are stored in the memory, and the instructions can cause the at least one processor to execute a method according to any one of the embodiments of the present disclosure, and an electronic device executed by the at least one processor is provided.

[0008] The sixth aspect provides a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute a method according to any one of the embodiments of the present disclosure.

Advantages of the Invention

[0009] The beneficial effects of the invention according to the present disclosure include at least the following. By comprehensively applying image processing and laser scanning technology, the accuracy and efficiency of yarn state detection can be effectively improved. By adopting an advanced machine learning method, the system can have better adaptability and intelligence, detect abnormal states at an early stage, and improve the stability of the entire spinning process and the quality of products.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Embodiments for Carrying Out the Invention

[0011] It should be understood that the content described in this part does not limit the key points or important features related to the embodiments of the present disclosure, nor does it limit the scope of the present disclosure. Other features of the present disclosure can be easily understood based on the following description.

[0012] In the drawings, unless otherwise specified, the same reference numerals in multiple drawings indicate the same or similar members or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only show some embodiments of the present disclosure and should not be regarded as limiting the scope of the present disclosure.

[0013] Hereinafter, the present invention will be described in more detail with reference to the drawings. In each figure, members having the same or similar functions are denoted by the same reference numerals. Although various aspects of the embodiments are shown in the drawings, unless otherwise specified, the drawings are not necessarily drawn to scale.

[0014] In addition, in order to better explain the present disclosure, many specific details are shown in the following specific embodiments. Those skilled in the art should understand that the present disclosure can be implemented similarly even without certain specific details. In some embodiments, methods, means, elements, circuits, etc. well known to those skilled in the art are not described in detail in order to emphasize the gist of the present disclosure.

[0015] In the related art, after the polyester melt is extruded from the spinneret, it cools and solidifies to form primary fibers. The primary fibers are oiled and bundled by an oiling nozzle, and then sent to a winder through a thread guide hook for winding and forming. After working for a certain period of time, the polyester material remaining around the yarn discharge holes of the spinneret gradually accumulates. After the remaining polyester material hardens, it blocks the yarn discharge holes, thus changing the fine cross-sectional shape of the melt, causing scratches on the fiber yarns, making the extruded yarns thinner, and potentially causing situations such as yarn floating and breakage. The conventional method is to perform blade cleaning on the spinneret at regular intervals to ensure the cleanliness of each yarn discharge hole in the spinneret. Such a regular maintenance policy lacks flexibility, may be performed frequently in some cases, wasting resources and time, or may be insufficient in some cases. Once yarn breakage occurs, the entire winding package has to be discarded.

[0016] Since the yarn extrusion speed is very fast, it is difficult to accurately capture the state of the yarn. Such blurring caused by high-speed movement not only affects the image quality but also reduces the accuracy of fault detection. The conventional technology cannot be visually detected unless obvious abnormal situations such as yarn floating and yarn breakage occur.

[0017] To at least partially solve one or more of the above-mentioned problems and other potential problems, embodiments of the present disclosure combine visual detection and laser scanning, reinforce image data and laser reflection data through an algorithm, and provide a method for detecting the yarn extrusion state that accurately detects the state of the yarn. According to the technical solution of the embodiments of the present disclosure, the accuracy and efficiency of yarn state detection can be effectively improved, abnormal states can be detected early, and the stability of the entire spinning process and the quality of the product can be improved.

[0018] FIG. 1 is a flowchart of a method for detecting the yarn extrusion state according to an embodiment of the present disclosure. As shown in FIG. 1, the method includes at least the following steps.

[0019] S110: Collect the first image of the target yarn path and laser reflection data inside the spinning container. In an embodiment of the present disclosure, the spinning container has a plurality of spinning nozzles and the same number of yarn paths. The yarn path can be understood as the movement path of a set of fiber yarn bundles corresponding to each spinning nozzle. Specifically, as shown in FIG. 2, the three first spinning nozzles 210 in the figure are located in the front row, and the two second spinning nozzles 220 are located in the rear row. The yarn path is the directly observable movement path of a set of fiber yarns 230 extruded from one spinning nozzle during the process of being oiled and bundled by the oil supply nozzle 240 and then entering the spinning cylinder through the yarn guide hook 250. Once it enters the spinning cylinder, it is difficult to observe. The fiber yarn 230 is ghost-like before being bundled. Note that the yarn guide hook may include an upper yarn guide hook and a lower yarn guide hook located above the oil supply nozzle.

[0020] The first image of the target yarn path is an image obtained by imaging at least one yarn path with a camera device. Multiple yarn paths can be imaged at once, and the first image of the target yarn path can be cut out from them. In order to obtain better image quality and a better imaging angle, each target yarn path may be imaged separately. The camera device may be a high-speed camera that uses high-speed imaging technology to capture yarn path images. The high-speed imaging camera can reduce or remove ghosts caused by the fast movement of the yarn bundle by capturing images at an extremely high frame rate. With appropriate light irradiation and imaging angle, the visibility of the yarn state can be enhanced.

[0021] The laser reflection data is the reflection signal data obtained by irradiating the yarn bundle of the target yarn path using laser scanning technology. When a laser beam irradiates a running yarn bundle, scattering, reflection, or refraction phenomena occur. The signals generated by these optical interactions provide important information about the yarn bundle. The collected reflection signal data can be processed by an algorithm to extract the parameter of the key point regarding the yarn state.

[0022] S120: Based on the first image, obtain the image features of the target yarn path. Using an image processing algorithm (e.g., edge detection, contrast enhancement), image features of the yarn in the target yarn path can be extracted, such as the width, shape, and distance between yarn bundles of the yarn.

[0023] S130: Based on the laser reflection data, obtain the laser characteristics of the target yarn path. By performing feature extraction on the laser reflection data, laser characteristics of the target yarn path can be obtained, such as diameter, speed, surface roughness, and optical characteristics due to laser scattering and reflection. By analyzing the distribution of the reflection intensity, the diameter and surface condition of the yarn can be estimated.

[0024] S140: Using a yarn extrusion state detection model, obtain the yarn extrusion state of the target yarn path based on the image features and laser characteristics. By using a pre-trained machine learning or deep learning model (e.g., convolutional neural network, support vector machine, etc.) to process the image features and laser characteristics, the yarn extrusion state of the yarn path is output. In one example, since a convolutional neural network (CNN) model is good at processing image data and can provide detailed predictions by probability output, the yarn extrusion state detection model can use a convolutional neural network (CNN) model.

[0025] The yarn extrusion state may include a plurality of state categories defined according to different possible situations and risk levels of the yarn. These state categories not only indicate the current situation of the yarn but also suggest the risk level that can lead to yarn breakage. The following is a set of exemplary states.

[0026] State 1 - Normal: Explanation: The running of the yarn is normal and no abnormality is visible. Risk level: Low. There is almost no risk of yarn breakage. State 2 - Slight fluctuation: Explanation: Slight waves or vibrations occur in the yarn, but they do not exceed the normal range. Risk level: Low-medium, indicating that fine-tuning to the machine's operating conditions may be necessary. State 3 - Obvious vibration Description: The yarn wave is obvious and may affect the yarn quality. Risk level: Medium. Long-term operation may cause quality issues or small probability of thread breakage. State 4 - Twisted: Description: The yarn surface is twisted or roughened, and the yarn path is irregular. Risk level: Medium to high. In this case, the possibility of yarn breakage and production stoppages increases. Condition 5-Tension Abnormal: Description: Thread tension is significantly abnormal and may indicate it is either too loose or too tight. Risk level: High. High possibility of thread breakage. Condition 6 - Obvious damage: Description: The threads show obvious signs of physical damage, such as wear and tear or signs of impending breakage. Risk level: Very high. Very likely to cause yarn breakage or severe product quality issues soon.

[0027] After obtaining the detection result of the yarn extrusion state, blade cleaning work can be performed in a timely manner on the spinning chamber corresponding to the yarn path with a high risk level according to the operator's situation and the production takt time.

[0028] According to the technical solution of the embodiment of the present disclosure, the accuracy and efficiency of yarn condition detection can be effectively improved by combining the application of image processing and laser scanning technologies, and the adoption of advanced machine learning methods can make the system more adaptable and intelligent, so as to detect abnormal conditions early, and improve the stability of the entire spinning process and the quality of the products.

[0029] In one possible embodiment, (S110) collecting a first image and laser reflection data of the target yarn path in the spinning vessel further comprises the following steps:

[0030] S111: Control to move the patrol device in front of the spinning container to be detected. In an embodiment of the present disclosure, the patrol device may be an AGV (Automated Guided Vehicle) operating according to a preset route or command, or a patrol device moving on a fixed track, but is not limited herein. The advantage of using the patrol device is that a set of collection devices can detect multiple spinning containers in multiple adjacent channels one by one. Moreover, the movable patrol device can adjust the imaging angle to collect better quality detection data.

[0031] S112: Trigger the camera device and laser scanning device of the patrol device, and collect the first image and laser reflection data of the target yarn path in the spinning container. The target yarn path includes a moving path in which a plurality of yarn bundles ejected from one spinning die are gathered into one by a yarn guide hook.

[0032] In an embodiment of the present disclosure, in order to avoid the front row yarn path shielding the rear row yarn path and to facilitate imaging the yarn paths at different positions in the rear row at an appropriate angle, a plurality of detection stations can be preset in front of the spinning container. That is, each detection station may be used to collect data of a plurality of yarn paths. After reaching the detection station, set the line corresponding to the detection station as a plurality of target yarn paths, trigger the camera device and laser scanning device of the patrol device, and sequentially collect the first image and laser reflection data for each target yarn path.

[0033] According to the technical solution of the embodiment of the present disclosure, by using a movable patrol device to collect data of the target yarn path at a plurality of detection stations, better quality detection data can be collected.

[0034] In a possible embodiment, obtaining the image features of the target yarn path based on the first image (S120) further includes the following steps. S121: Perform morphological processing on the first image to obtain a second image. S122: Obtain the image features of the target yarn path based on the second image. In the embodiments of the present disclosure, by using morphological transformations (e.g., erosion, dilation) to improve the image composition of the yarn, the yarn state features in the obtained second image can be more easily recognized by the model.

[0035] In a possible embodiment, S121 performing morphological processing on the first image to obtain a second image further includes the following steps. Perform dilation operations at the sliding degree across the first image using a first sliding window to obtain a dilated image. Perform erosion operations at the sliding degree across the first image using a second sliding window to obtain an eroded image. Obtain the second image based on the morphological gradient of the dilated image and the eroded image.

[0036] In the embodiments of the present disclosure, the dilation operation is a morphological operation that has the effect of expanding the objects in the image. Specifically, using a single structuring element as a sliding window to move across the image, when it overlaps with any part of the object (yarn), pixels can be added to the center position of the structuring element. As a result, the boundaries of the object expand outward, and larger structures are emphasized.

[0037] Erosion is an operation opposite to dilation that has the effect of shrinking the objects in the image. Using a "sliding window", the rule is to keep the center pixel of the window unchanged only when all pixels within the window are part of the object. As a result, the boundaries of the object shrink inward, and details and small objects can be removed.

[0038] The morphological gradient is calculated by subtracting the image after shrinking from the image after dilation. Since the edges of the object are enlarged by dilation and reduced by shrinking, they are emphasized by the difference between the two. Such a method is particularly suitable for emphasizing thin lines and details such as fiber edges in the thread image.

[0039] According to the technical solution of the embodiments of the present disclosure, by applying a morphological gradient to the thread image, the edges of the thread can be made clearer and more prominent. This contributes significantly to subsequent feature extraction and model prediction. Thereby, in order to provide a clearer thread contour, the algorithm can more easily distinguish the thread from the background. Such edge enhancement technology is particularly advantageous for processing thread images generated under high-speed movement or low-contrast conditions, and strengthens the identification ability of the model for the thread state.

[0040] In a possible implementation form, the first sliding window and the second sliding window adopt different structural elements, and the vicinity and shape of the structural element (or kernel) depend on the specific features to be extracted or emphasized from the image. After being extruded from the spinneret, the fiber yarn bundles gradually gather into bundles from top to bottom, that is, the yarn path is substantially in the vertical direction. When processing morphologically, the thickness of the yarn and the degree of surface roughness should be emphasized. Therefore, the constituent elements used for shrinking and dilation need to be designed according to the specific characteristics of the yarn. Considering that the yarn is substantially in the vertical direction and the need to emphasize the horizontal fluff and burrs, the structural elements can be designed according to the following policy.

[0041] As the structural element for shrinking, in order to emphasize the longitudinal characteristics of the yarn and reduce the influence of burrs, a rectangular structural element that is long in the vertical direction can be used. Dimensionally, the height of the structural element needs to be greater than the width, so that the horizontal characteristics of the yarn can have more influence in the shrinking process. The constituent element obtained in this way contributes to reducing the influence of burrs by removing small horizontal protrusions of the thread edge during shrinking.

[0042] For the structural elements related to expansion, similar rectangular structural elements can be used. To recover the longitudinal information of the yarn lost due to contraction, the width of the structural elements related to expansion may be slightly larger, but they remain relatively long in the vertical direction. The obtained structural elements contribute to the recovery of the original thickness of the yarn during the expansion process and avoid excessive expansion of the horizontal burrs.

[0043] As an example, the first sliding window is a rectangular structural element with an aspect ratio of 3:2 to 5:2, and the second sliding window is a rectangular structural element with an aspect ratio of 2:1 to 5:1. For example, as shown in Figure 3, the first sliding window has a length of 9*X pixels and a width of 5*X pixels. The second sliding window has a length of 9*X pixels and a width of 3*X pixels. Both are in the vertical direction as the longitudinal direction, and X is a positive integer greater than 0.

[0044] In another example, the first sliding window is an inverted trapezoidal structural element with an aspect ratio of 3:2 to 5:2 and wider at the top and narrower at the bottom, and the ratio of the upper and lower widths is 1.1:1 to 1.3:1. The second sliding window is an inverted trapezoidal structural element with an aspect ratio of 2:1 to 5:1, and the ratio of the upper and lower widths is 1.1:1 to 1.3:1. The inverted trapezoidal structure is used to provide a shape that is wider at the top and narrower at the bottom, approximating the yarn path.

[0045] It should be noted that the selection of the structural elements needs to be adjusted according to the specific application scenario and the desired results. The optimal size and shape may need to be determined by experiments.

[0046] According to the technical solution of the embodiments of the present disclosure, the structural elements designed in the above manner can effectively emphasize the thickness and surface roughness of the yarn and reduce the influence of horizontal burrs. The design of this customized structural element is a key part of the morphological processing and can significantly improve the effect of yarn image processing.

[0047] In one possible implementation, obtaining the image features of the target thread path by S122 based on the second image further includes the following steps. S122-1: Divide the second image into multiple segments along the longitudinal direction of the target thread path to obtain multiple segmented sub-images. S122-2: Perform feature extraction on the multiple segmented sub-images to obtain a plurality of first image features. S122-3: Obtain the image features of the target thread path based on the plurality of first image features and corresponding weight values.

[0048] In the embodiments of the present disclosure, since the thread path gradually converges and narrows from top to bottom and finally converges into one, the closer to the bottom, the smaller the interval between each fiber thread bundle, and it becomes more difficult to accurately identify the thickness and surface roughness of each thread bundle. Therefore, along the thread path direction, higher weight values can be sequentially set for the first image features corresponding to the sub-images of each segment. That is, higher weight values are assigned to the first image features extracted from the multiple sub-images in the upper part of the thread path, and the weight values of the first image features extracted from the sub-images in the lower part of the thread path are lowered.

[0049] According to the technical solution of the embodiments of the present disclosure, by assigning corresponding weight values to the first image features extracted from the sub-images of each segment based on the morphological features of the thread path, it contributes to making the model pay more attention to the image features in the upper region of the thread path.

[0050] In laser scanning detection, threads in different thread extrusion states exhibit obvious differences in laser reflection or scattering data. These differences mainly result from changes in the physical properties and motion states of the threads, specifically including the following.

[0051] (1) Thread diameter: Threads of different specifications may have different diameters, and the pore diameters of the micropores of the spinneret used are different. For example, the diameter of the thread in the normal state should be uniform and consistent. On the other hand, as the thread extrusion hole is gradually blocked, the diameter of the thread can change abnormally. (2) Surface roughness: In the normal state, the yarn is usually smooth on the surface, while the defective yarn may exhibit high surface roughness, which can affect the scattering pattern of the laser beam. (3) Scattering intensity: The scattering intensity changes as the surface properties of the yarn change. On a smooth surface, the scattering signal is weak and uniform, while on a rough or irregular surface, it causes amplification and pattern changes of the scattering signal. (4) Scattering angle: During laser scanning, the scattering angles of yarns in different states may vary due to differences in surface characteristics and shapes. For example, the scattering angle of normal yarn is relatively stable, while that of abnormal yarn (e.g., with changes in surface roughness or cross-sectional shape) may become irregular. (5) Spectral characteristics: The spectral characteristics generated when the laser light interacts with the yarn may also vary depending on the state of the yarn. Depending on the physical state, the spectrum of the laser light may change slightly. (6) Continuity of the reflected beam: The reflection of normal yarn is continuous and uniform during laser scanning, while discontinuous yarn may cause interruptions or irregularities in the reflection.

[0052] In short, by analyzing the differences in these laser scanning data, the abnormal characteristics of the fiber can be effectively identified and distinguished, providing important information for the monitoring and quality control of the spinning process.

[0053] In a possible implementation form, obtaining the image features of the target yarn path by S122-3 based on a plurality of first image features and corresponding weight values further includes the following steps. Determine the weight value of the multi-stage sub-image based on the width value of the yarn path included in the multi-stage sub-image. Obtain the second image features of the multi-stage sub-image based on the weight value and the first image features of the multi-stage sub-image. Obtain the image features of the target yarn path based on the second image features of the multi-stage sub-image.

[0054] In an embodiment of the present disclosure, based on the value of the width of the thread path in each stage of the sub-image, the corresponding weight value can be set. The value of the width may be the maximum width, the minimum width, or the average width. Multiply the weight value by the first image feature to obtain the second image feature. Further, integrate the second image features of the multi-stage sub-images to finally obtain the image feature of the target thread path.

[0055] In a possible implementation, obtaining the laser feature of the target thread path based on the laser reflection data in S130 further includes the following steps.

[0056] S131: Perform a short-time Fourier transform on the laser reflection data to obtain a time-frequency representation result. In an embodiment of the present disclosure, a short-time Fourier transform (STFT) is performed on the laser reflection data, a sliding window is applied to the signal, and a Fourier transform is performed on the signal within each window to obtain the time-frequency representation of the signal. The time locality is guaranteed, and the frequency components of the signal over time can be observed.

[0057] S132: Determine the time-frequency feature based on the time-frequency representation result. The time-frequency feature of the laser reflection data is analyzed using the result of the STFT, that is, the time-frequency graph. The time-frequency graph shows the change over time of different frequency components of the signal, provides the frequency information of the signal at different time points, and the time-frequency features to be identified specifically include the following.

[0058] Identifying the distribution of frequency components in the time-frequency graph includes confirming which frequencies are dominant in the signal and how these frequencies change over time. For example, in the laser scanning data of the thread, the prominence of a certain frequency may indicate a specific physical state or change.

[0059] Analyze the energy distribution and evaluate the energy of frequency components at different times. In the time-frequency graph, regions with high energy usually appear as brighter regions. The energy distribution can indicate the dynamic characteristics of the signal, such as the stability of the thread or the occurrence of abnormal conditions.

[0060] Observe the time change and pay attention to the change of frequency components over time. For example, a sudden change in frequency may indicate a change in the thread extrusion state or some abnormality.

[0061] S133: Obtain the laser characteristics of the target thread path based on the time-frequency characteristics. In the embodiments of the present disclosure, by further operating and analyzing the time-frequency characteristics obtained from the time-frequency graph, important information is extracted to obtain the laser characteristics of the target thread path. Specifically, it may include identifying the energy peaks within a specific frequency range, calculating the duration of different frequency components, etc.

[0062] According to the technical solution of the embodiments of the present disclosure, rich information can be extracted from the time-frequency graph, which contributes to accurately identifying the behavior and state of the thread during laser scanning and realizing quality control.

[0063] In a possible implementation form, S133 obtaining the laser characteristics of the target thread path based on the time-frequency characteristics further includes the following steps. Extract the main frequency characteristics, energy peak characteristics, and dynamic change characteristics from the time-frequency characteristics. Obtain the laser characteristics of the target thread path based on the main frequency characteristics, energy peak characteristics, and dynamic change characteristics.

[0064] In the embodiments of the present disclosure, by further performing feature extraction on the time-frequency characteristics, the extracted features should capture the important attributes of the signal, and the specific steps include the following. Extract the main frequency components and confirm the specific values of the main frequencies, their durations, and intensities in the entire signal. In the time-frequency graph, energy peaks indicating that the signal intensity is particularly high at specific times and frequencies are identified. Recording the position (time and frequency) and magnitude of these peaks contributes to identifying specific features or anomalies in the signal. Analyze the pattern of change over time of the frequency components. Pay attention to how the frequency components increase, decrease, or remain stable over time. Any sudden changes or irregular patterns in the identified frequencies may indicate an abnormal state of the thread.

[0065] Through the above steps, a plurality of effective time-frequency features can be extracted from the laser signal and used as laser features.

[0066] In one possible implementation, S140 uses a thread extrusion state detection model to obtain the thread extrusion state of the target thread path based on the image features and laser features, which further includes the following steps.

[0067] S141: Based on the image features and laser features, obtain the thread extrusion state features of the target thread path by fusion. S142: Using the thread extrusion state detection model, obtain the thread extrusion state of the target thread path based on the thread extrusion state features.

[0068] In the embodiments of the present disclosure, data fusion technology can be used to integrate image features and laser features, integrate data sets from two different sources, and combine two groups of features into one single feature vector. The thread extrusion state detection model is pre-trained, and by training a machine learning model, various state categories of the thread extrusion state can be identified. This requires a certain number of thread path image data for training the model, and the trained thread extrusion state detection model can quickly and accurately identify the thread extrusion state and maintain high accuracy even under complex or changing conditions.

[0069] The main types of yarns related to the technical solutions of the embodiments of the present disclosure may include one or more of partially oriented yarns (POY), fully drawn yarns (FDY), draw textured yarns (DTY) (or also called low stretch yarns). For example, the types of yarns may specifically include, for example, polyester partially oriented yarns, polyester fully drawn yarns, polyester drawn yarns, polyester draw textured yarns, etc.

[0070] The present disclosure further provides a method for training a yarn extrusion state detection model. FIG. 4 is a flowchart of the method for training a yarn extrusion state detection model. As shown in FIG. 4, the method includes the following steps.

[0071] S410: Obtain training image features of the target yarn path based on the sample image of the target yarn path. S420: Obtain training laser features of the target yarn path based on the laser sample data of the target yarn path. S430: Use the first detection model to obtain a prediction result of the yarn extrusion state of the target yarn path based on the training image features and the training laser features. S440: Determine a loss function based on the prediction result of the yarn extrusion state of the target yarn path and the true result. S450: Update the parameters of the first detection model based on the loss function to obtain a trained yarn extrusion state detection model.

[0072] In one example, the structure of the first detection model includes an input layer, a convolutional layer, a pooling layer, a fusion layer, a fully connected layer, and an output layer.

[0073] The input layer receives the processed image data and laser reflection data, and these data may need to be subjected to appropriate conversion and normalization to adapt to the network input. Multiple convolutional layers can effectively extract the spatial features in the image. Each convolutional layer extracts the features of different layers by a set of learnable filters. The activation function (e.g., ReLU) is used to enhance the non-linearity of the network. The pooling layer reduces the number of parameters and the amount of computation by adding a pooling layer (e.g., max pooling layer) after the convolutional layer to reduce the spatial size of the feature map. The fusion layer sets one fusion layer after convolution and pooling to combine the features from the image and laser data. This can be achieved by stitching or other fusion techniques. One or more fully connected layers are used at the end of the network to integrate the previously extracted features. In the fully connected layer, the network model learns how to classify based on the extracted features. The output layer can provide the probability distribution of multiple state levels using the softmax activation function. The output of each neuron represents the probability that the thread is at a specific state level.

[0074] In a possible implementation, the sample image and laser sample data of the target thread path may be pre-collected and manually labeled. They may also be the normal images and laser reflection data collected by the patrol device during the daily patrol process. After a thread break occurs in the take-up machine or a quality problem is detected in the winding package in the subsequent appearance inspection flow and its level decreases, the corresponding image data and laser data collected during the patrol can be traced back and used as the sample data with abnormal problems.

[0075] In a possible implementation, obtaining the image features of the target thread path based on the sample image in S410 includes the following. Perform morphological processing on the sample image to obtain a third image. Based on the third image, obtain the training image features of the target thread path.

[0076] In a possible implementation form, obtaining the third image by performing morphological processing on the sample image includes the following. Using the first sliding window, perform dilation operations along the sample image according to the sliding degree to obtain a dilated image. Using the second sliding window, perform contraction operations along the sample image according to the sliding degree to obtain a contracted image. Based on the morphological gradient of the dilated image and the contracted image, obtain the third image.

[0077] In a possible implementation form, obtaining the training image features of the target thread path based on the third image is Segment the third image in multiple stages along the longitudinal direction of the target thread path to obtain multi-stage sample sub-images. Perform feature extraction on the multi-stage sample sub-images to obtain a plurality of sample sub-image features. Based on the plurality of sample sub-image features and the corresponding weight values, obtain the first training image features of the target thread path.

[0078] In a possible implementation form, obtaining the first training image features of the target thread path based on the plurality of sample sub-image features and the corresponding weight values is Based on the values of the widths of the thread paths included in the plurality of sample sub-images, determine the weight values of the multi-stage sample sub-images. Based on the weight values and the training image features of the multi-stage sample sub-images, obtain the second training image features of the multi-stage sample sub-images. Based on the second training image features of the multi-stage sub-images, obtain the training image features of the target thread path.

[0079] In a possible implementation form, S420 obtaining the training laser features of the target thread path based on the laser sample data of the target thread path includes the following. Perform a short-time Fourier transform on the laser sample data to obtain a time-frequency representation result. Based on the time-frequency representation result, determine the time-frequency features. Based on the time-frequency features, obtain the training laser features of the target thread path.

[0080] In a possible implementation form, obtaining the training laser features of the target thread path based on the time-frequency features includes the following. Extract the main frequency feature, energy peak feature, and dynamic change feature from the time-frequency features. Based on the main frequency feature, energy peak feature, and dynamic change feature, obtain the training laser features of the target thread path.

[0081] In a possible implementation form, for S430 to obtain the prediction result of the thread extrusion state of the target thread path using the first detection model based on the training image features and training laser features includes the following. Based on the training image features and training laser features, obtain the training state features of the target thread path through fusion. Using the first detection model, obtain the prediction result of the thread extrusion state of the target thread path based on the training state features.

[0082] In a possible implementation form, for S440 to determine the loss function based on the prediction result and the actual result of the thread extrusion state of the target thread path includes the following. In each iteration, the first detection model predicts the input training image features and training laser features and outputs the probability of each thread extrusion state. Compare the prediction result of the model with the actual thread state (actual label), and calculate the loss value based on the loss function. For a multi-class classification problem, usually, the cross-entropy loss function is used. The cross-entropy loss function can quantify the difference between the probability distribution of the model prediction and the actual label.

[0083] In a possible implementation form, for S450 to update the parameters of the first detection model based on the loss function to obtain a trained thread extrusion state detection model includes the following. The entire training dataset is divided into a plurality of batches, and for each batch, the parameters of the model are updated based on the gradient of the loss function using an optimizer. This process usually repeats multiple epochs (the number of times through the complete dataset). After each epoch, the performance of the model usually improves. Through the above process, a trained yarn extrusion state detection model can be obtained, and the model can predict the state of the yarn based on image features and laser features.

[0084] FIG. 5 is a schematic configuration diagram of a yarn extrusion state detection device according to an embodiment of the present disclosure. As shown in FIG. 5, the device includes at least a collection module 501, an image feature determination module 502, a laser feature determination module 503, and a detection module 504.

[0085] The collection module 501 collects a first image of the target yarn path in the spinning container and laser reflection data. The image feature determination module 502 obtains the image features of the target yarn path based on the first image. The laser feature determination module 503 obtains the laser features of the target yarn path based on the laser reflection data. The detection module 504 uses the yarn extrusion state detection model to obtain the yarn extrusion state of the target yarn path based on the image features and laser features.

[0086] In a possible implementation form, the collection module 501 is used for the following. Control the patrol device to move in front of the spinning container to be detected. Trigger the camera device and the laser scanning device of the patrol device, and collect a first image of the target yarn path in the spinning container and laser reflection data. The target yarn path includes a moving path in which a plurality of yarn bundles ejected from one spinneret are bundled into one by a yarn guide hook.

[0087] In a possible implementation form, the image feature determination module 502 is used for the following purposes. Perform morphological processing on the first image to obtain a second image. Obtain the image features of the target yarn path based on the second image.

[0088] In a possible implementation form, the image feature determination module 502 is used for the following purposes. Perform dilation operations along the first image using the first sliding window to obtain a dilated image. Perform erosion operations along the first image using the second sliding window to obtain an eroded image. Obtain the second image based on the morphological gradient of the dilated image and the eroded image.

[0089] In a possible implementation form, the image feature determination module 502 is used for the following purposes. Divide the second image into multiple segments along the longitudinal direction of the target yarn path to obtain multi-segment sub-images. Extract features from the multi-segment sub-images to obtain a plurality of first image features. Obtain the image features of the target yarn path based on the plurality of first image features and the corresponding weight values.

[0090] In a possible implementation form, the image feature determination module 502 is used for the following purposes. Determine the weight values of the multi-segment sub-images based on the values of the widths of the yarn paths included in the multi-segment sub-images. Obtain the second image features of the multi-segment sub-images based on the weight values of the multi-segment sub-images and the first image features. Obtain the image features of the target yarn path based on the second image features of the multi-segment sub-images.

[0091] In a possible implementation form, the laser feature determination module 503 is used for the following purposes. Perform a short-time Fourier transform on the laser reflection data to obtain a time-frequency representation result. Determine time-frequency features based on the time-frequency representation result. Based on the time-frequency characteristics, obtain the laser characteristics of the target yarn path.

[0092] In a possible implementation form, the laser characteristic determination module 503 is used for the following. Extract the main frequency characteristic, energy peak characteristic, and dynamic change characteristic from the time-frequency characteristics. Based on the main frequency characteristic, energy peak characteristic, and dynamic change characteristic, obtain the laser characteristics of the target yarn path.

[0093] In a possible implementation form, the detection module 504 is used for the following. Based on the image characteristics and laser characteristics, obtain the yarn extrusion state characteristics of the target yarn path through fusion. Using the yarn extrusion state detection model, obtain the yarn extrusion state of the target yarn path based on the yarn extrusion state characteristics.

[0094] For the specific functions and implementation forms of each module and sub-module of the device according to the embodiments of the present disclosure, reference can be made to the relevant descriptions of the corresponding steps in the above method embodiments, but the descriptions will not be repeated here.

[0095] FIG. 6 is a schematic structural diagram of a training device for a yarn extrusion state detection model. As shown in FIG. 6, the device includes an image feature extraction module 601 for obtaining the training image features of the target yarn path based on the sample image of the target yarn path, and a laser feature extraction module 602 for obtaining the training laser features of the target yarn path based on the laser sample data of the target yarn path, and a prediction module 603 for obtaining the prediction result of the yarn extrusion state of the target yarn path based on the training image features and training laser features using the first detection model, and a loss function determination module 604 for determining the loss function based on the prediction result of the yarn extrusion state of the target yarn path and the true result. A training module 605 for updating the parameters of the first detection model based on a loss function to obtain a trained yarn extrusion state detection model.

[0096] For the specific functions and embodiments of each module and sub-module of the device according to the embodiments of the present disclosure, reference may be made to the relevant descriptions of the corresponding steps in the above method embodiments, and the descriptions will not be repeated here.

[0097] FIG. 7 is a block diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 7, the electronic device includes a memory 710 and a processor 720. The memory 710 stores a computer program executable by the processor 720. The number of the memory 710 and the processor 720 may be one or more. The memory 710 can store one or more computer programs, and when the one or more computer programs are executed by the electronic device, the electronic device is caused to execute the method according to the above method embodiments. The electronic device may further include a communication interface 730 for communicating with an external device and performing data interaction transmission.

[0098] When the memory 710, the processor 720, and the communication interface 730 are independently realized, they are connected to each other via a bus and can communicate with each other. The bus may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For convenience, in FIG. 7, only one thick line is shown, but it does not mean that there is only one bus or one type of bus.

[0099] Alternatively, when specifically implemented, if the memory 710, the processor 720, and the communication interface 730 are integrated on a single chip, they can communicate with each other via an internal interface.

[0100] The processor may be a Central Processing Unit (CPU), or may be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components, etc. It should be understood that the general-purpose processor may be a microprocessor or any conventional processor, etc. Additionally, the processor may be a processor that supports the Advanced RISC Machine (ARM) architecture.

[0101] Furthermore, optionally, the memory may include a read-only memory and a random access memory, and may further include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both a volatile memory and a non-volatile memory. The non-volatile memory may include a ROM (Read-Only Memory), PROM (Programmable ROM), EPROM (Erasable PROM), EEPROM (Electrically EPROM), or flash memory. The volatile memory may include a random access memory (Random Access Memory, RAM) used as an external cache. The above description is merely illustrative and not restrictive. Many forms of RAM can be used. For example, static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic Random Access Memory, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Date SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchlink dynamic random access memory (Synchlink DRAM, SLDRAM), and direct memory bus random access memory (Direct RAM BUS RAM, DR RAM).

[0102] In the above embodiments, all or part of them may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of it can be realized in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer loads and executes the computer instructions, all or part of the flow or functions described in the embodiments of the present disclosure are generated. The computer may be a general-purpose computer, a dedicated computer, a computer network, or another programmable device. The computer instructions may be stored in a computer-readable storage medium or may be transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, Bluetooth (registered trademark), microwave, etc.). The computer-readable storage medium may be any available medium accessible by a computer or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (e.g., a floppy (registered trademark) disk, a hard disk, or a magnetic tape), an optical medium (e.g., a Digital Versatile Disc (DVD)), a semiconductor medium (e.g., a Solid State Disk (SSD)), etc. It should be noted that the computer-readable storage medium according to the present disclosure may be a non-volatile storage medium, in other words, a non-temporary storage medium.

[0103] A person skilled in the art would understand that all or part of the steps for implementing the above embodiments may be executed by hardware, or may be completed by instructing the relevant hardware through a program, and the program may be stored in a computer-readable storage medium, and the above storage medium may be a read-only memory, a magnetic disk, an optical disk, etc.

[0104] In the description of the embodiments of the present disclosure, the descriptions of the terms "one embodiment", "some embodiments", "exemplification", "specific exemplification", or "some exemplifications" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or exemplification are included in at least one embodiment or exemplification of the present disclosure. Further, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or exemplifications in an appropriate manner. Also, unless they are inconsistent with each other, a person skilled in the art can combine the different embodiments or exemplifications described in this specification and the features in different embodiments or exemplifications.

[0105] In the description of the embodiments of the present disclosure, unless otherwise specified, " / " means "or". For example, "A / B" can represent "A" or "B". The "and / or" in this specification is only a relationship for describing the relevant objects, indicating the possibility of the existence of three types of relationships. For example, "A and / or B" can indicate three situations: "A" exists alone, "A" and "B" exist simultaneously, and "B" exists alone.

[0106] In the description of the embodiments of the present disclosure, the terms "first", "second", etc. are for description only and should not be understood as indicating or implying relative importance or indicating the number of the indicated components. Thus, the features limited by "first", "second" can include one or more of such features explicitly or implicitly. In the description of the embodiments of the present disclosure, unless otherwise specified, the meaning of "a plurality" is two or more.

[0107] The above are only exemplary embodiments of the present disclosure and do not limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made in accordance with the spirit and principles of the present disclosure should all be included within the protection scope of the present disclosure.

Claims

1. Collecting a first image and laser reflection data of a target yarn path in a spinning container; Performing morphological processing on the first image to obtain a second image; Dividing the second image into multiple stages along the length direction of the target yarn path to obtain multi-stage sub-images; Performing feature extraction on the multi-stage sub-images to obtain a plurality of first image features; Obtaining the image features of the target yarn path based on the plurality of first image features and corresponding weight values; Performing a short-time Fourier transform on the laser reflection data to obtain a time-frequency representation result; Determining time-frequency features based on the time-frequency representation result; Obtaining the laser features of the target yarn path based on the time-frequency features; Using a yarn extrusion state detection model to obtain the state category of the target yarn path based on the image features and the laser features, including: The weight value is determined based on the value of the width of the yarn path included in the multi-stage sub-images, A method for detecting the yarn extrusion state.

2. Collecting a first image and laser reflection data of a target yarn path in a spinning container includes: Controlling to move in front of the spinning container to be detected by the patrol device; Triggering the camera device and the laser scanning device of the patrol device, and collecting a first image and laser reflection data of the target yarn path in the spinning container, The target yarn path includes a moving path in which a plurality of yarn bundles ejected from one spinneret are bundled into one through a yarn guide hook. The method according to Claim 1.

3. Performing morphological processing on the first image to obtain a second image includes: Using a first sliding window to perform dilation operation on the first image according to the sliding degree to obtain a dilated image; Using a second sliding window to perform erosion operation on the first image according to the sliding degree to obtain an eroded image; Obtaining a second image based on the morphological gradient of the dilated image and the eroded image. The method according to Claim 1.

4. Obtaining the image features of the target yarn path based on the plurality of first image features and corresponding weight values includes: Determining the weight values of the multi-stage sub-images based on the values of the widths of the yarn paths included in the multi-stage sub-images; Obtaining the second image features of the multi-stage sub-images based on the weight values of the multi-stage sub-images and the first image features. Obtaining the image features of the target thread path based on the second image features of the multi-stage sub-images, The method according to claim 1.

5. Obtaining the laser features of the target thread path based on the time-frequency features includes: Extracting the main frequency feature, energy peak feature, and dynamic change feature from the time-frequency features, Obtaining the laser features of the target thread path based on the main frequency feature, the energy peak feature, and the dynamic change feature, The method according to claim 1.

6. Using the thread extrusion state detection model to obtain the thread extrusion state of the target thread path based on the image features and the laser features includes: Obtaining the thread extrusion state features of the target thread path by fusion based on the image features and the laser features, Using the thread extrusion state detection model to obtain the thread extrusion state of the target thread path based on the thread extrusion state features, The method according to claim 1.

7. Performing morphological processing on the sample image to obtain a third image, Dividing the third image into multiple stages along the length direction of the target thread path to obtain multi-stage sample sub-images, Performing feature extraction on the multi-stage sub-images to obtain a plurality of sample sub-image features, Obtaining the training image features of the target thread path based on the plurality of sample sub-image features and the corresponding weight values, Performing a short-time Fourier transform on the laser sample data to obtain a time-frequency representation result, Determining the time-frequency features based on the time-frequency representation result, Obtaining the training laser features of the target thread path based on the time-frequency features, Using the first detection model to obtain the prediction result of the state category of the target thread path based on the training image features and the training laser features, Determining the loss function based on the prediction result of the state category of the target thread path and the true result, Updating the parameters of the first detection model based on the loss function to obtain a trained thread extrusion state detection model, The weight value is determined based on the value of the width of the thread path included in the multi-stage sub-image, A training method for a thread extrusion state detection model.

8. A collection module for collecting the first image and laser reflection data of the target thread path in the spinning container, Performing morphological processing on the first image to obtain a second image, dividing the second image into multiple stages along the length direction of the target yarn path to obtain multi-stage sub-images, performing feature extraction on the multi-stage sub-images to obtain a plurality of first image features, and obtaining the image features of the target yarn path based on the plurality of first image features and corresponding weight values, an image feature determination module; Performing short-time Fourier transform on the laser reflection data to obtain a time-frequency representation result, determining time-frequency features based on the time-frequency representation result, and obtaining the laser features of the target yarn path based on the time-frequency features, a laser feature determination module; A detection module for obtaining the state category of the target yarn path based on the image features and the laser features by using a yarn extrusion state detection model; The weight value is determined based on the value of the width of the yarn path included in the multi-stage sub-images. A device for detecting the yarn extrusion state.

9. The collection module: Controlling to move in front of the spinning container to be detected by the patrol device, Triggering the camera device and the laser scanning device of the patrol device, and used to collect the first image and the laser reflection data of the target yarn path in the spinning container. The target yarn path includes a moving path in which a plurality of yarn bundles ejected from one spinning die are bundled into one through a yarn guide hook. The device for detecting the yarn extrusion state according to claim 8.

10. The image feature determination module: Using a first sliding window to perform dilation operation on the first image according to the sliding degree to obtain a dilated image. Using a second sliding window to perform erosion operation on the first image according to the sliding degree to obtain an eroded image. Used to obtain a second image based on the morphological gradient of the dilated image and the eroded image. The device for detecting the yarn extrusion state according to claim 8.

11. The image feature determination module: Determining the weight value of the multi-stage sub-images based on the value of the width of the yarn path included in the multi-stage sub-images. Obtaining the second image features of the multi-stage sub-images based on the weight value of the multi-stage sub-images and the first image features. Used to obtain the image features of the target yarn path based on the second image features of the multi-stage sub-images. The device for detecting the yarn extrusion state according to claim 8.

12. The laser feature determination module: Extract the main frequency feature, energy peak feature, and dynamic change feature from the time-frequency feature, Based on the main frequency feature, the energy peak feature, and the dynamic change feature, it is used to obtain the laser feature of the target yarn path. The yarn extrusion state detection device according to claim 8.

13. The detection module is Based on the image feature and the laser feature, obtain the yarn extrusion state feature of the target yarn path by fusion. Using a yarn extrusion state detection model, based on the yarn extrusion state feature, it is used to obtain the yarn extrusion state of the target yarn path. The yarn extrusion state detection device according to claim 8.

14. Perform morphological processing on the sample image to obtain a third image, divide the third image into multiple segments along the length direction of the target yarn path to obtain multiple segmented sample sub-images, perform feature extraction on the multiple segmented sample sub-images to obtain multiple sample sub-image features, and based on the multiple sample sub-image features and corresponding weight values, an image feature extraction module for obtaining the training image feature of the target yarn path, Perform short-time Fourier transform on the laser sample data to obtain a time-frequency representation result, determine the time-frequency feature based on the time-frequency representation result, and based on the time-frequency feature, a laser feature extraction module for obtaining the training laser feature of the target yarn path, Using a first detection model, a prediction module for obtaining a prediction result of the state category of the target yarn path based on the training image feature and the training laser feature, A loss function determination module for determining a loss function based on the prediction result of the state category of the target yarn path and the true result, Based on the loss function, update the parameters of the first detection model to obtain a trained yarn extrusion state detection model, and a training module. The weight value is determined based on the value of the width of the yarn path included in the multiple segmented sub-images. A training device for the yarn extrusion state detection model.

15. At least one processor, A memory communicably connected to at least one processor, The memory stores instructions executable by the at least one processor, and the instructions can be executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 7. An electronic device.

16. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the method according to any one of claims 1 to 7, wherein the computer-readable storage medium is non-transitory.

Images