Straw defect detection and production parameter automatic control method
By using a multi-task learning model and deep neural networks to monitor straw defects in real time, the problem of weak generalization ability of traditional detection models and lag in manual parameter tuning has been solved, realizing automatic optimization of straw production parameters and efficient detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-09
- Publication Date
- 2026-04-07
AI Technical Summary
In existing technologies, defect detection models in straw production processes have weak generalization capabilities, high false detection rates, and manual parameter tuning is slow and costly, making it impossible to respond to production changes in real time.
A multi-task learning model is used for straw defect detection and automatic control of production parameters. The model is trained by image acquisition and multi-labeling, and combined with deep neural networks and LSTM networks to monitor and optimize production parameters in real time.
It improves the accuracy and versatility of defect detection, reduces the false detection rate, reduces the lag and cost of manual adjustment, and improves product quality stability.
Smart Images
Figure CN121810583A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of industrial defect detection and automatic control, and in particular to a method for detecting straw defects and automatically controlling production parameters. Background Technology
[0002] Straws, as a daily disposable product, are widely used in industries such as catering and healthcare, resulting in a huge market demand. During the straw production process, various defects inevitably occur due to factors such as the temperature and extrusion speed of the production equipment, including impurities on the straw wall, wrinkles, yellow spots on the inner wall, and uneven diameter. These defects not only affect the appearance and performance of the straw but may also pose a potential threat to the user's health. Therefore, rigorous quality testing of straws is a crucial step in ensuring product quality.
[0003] In recent years, the rapid development of deep learning technology has brought new ideas to the visual inspection of straw defects. Deep learning-based visual inspection can train and identify different types of defects separately, effectively reducing false detections caused by various interference factors. However, traditional visual inspection methods such as single-model classification and defect localization require training separate models for each type of straw product, resulting in weak generalization ability, insufficient transferability, and inadequate recognition accuracy of deep learning models. Furthermore, they can only determine the type and location of defects, but cannot accurately output parameters such as defect size.
[0004] On the other hand, even if straw defects are detected by inspection equipment, defective products can only be passively rejected, increasing average production costs. Furthermore, when frequent defects are detected by the inspection equipment, manual parameter adjustments by on-site staff are necessary. However, manual parameter adjustments heavily rely on employee experience and cannot respond promptly to changes in production equipment and raw materials. Additionally, since straw production primarily involves extruding raw materials after melting them at high temperatures, the internal environment of the equipment undergoes a series of changes during long-term continuous operation, rendering preset temperatures and extrusion speeds unsuitable, leading to increased product defects and a decreased yield. Manual adjustments often cannot respond to these changes in a timely manner, and real-time manual parameter adjustments require significant manpower, increasing labor costs. Summary of the Invention
[0005] The technical problem this invention aims to solve is to address the shortcomings of existing technologies by providing a method for straw defect detection and automatic control of production parameters. This method uses a multi-task model with simultaneous classification, detection, and segmentation functions to replace the traditional single-task model. The single model achieves straw type classification, defect location detection, and defect size segmentation of the input image, thus solving the problems of cumbersome model switching, large sample requirements, and high false positive and false negative rates in traditional single-model deep learning detection algorithms. Simultaneously, by monitoring defect conditions and production parameters in real time over a period of time, and extracting defect and production parameter features through a deep neural network, the optimal production parameters are automatically calculated, achieving real-time automatic adjustment of straw production parameters.
[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:
[0007] A method for detecting straw defects and automatically controlling production parameters includes the following steps:
[0008] Step 1: Manually adjust various production control parameters during the production process, while simultaneously acquiring straw images and recording the production control parameter data at the time of image acquisition. Perform multiple annotations on the acquired straw images to establish a composite sample set containing straw image samples and production parameters.
[0009] Step 2: Input the composite sample set into the defect detection and automatic control multi-task model for training to obtain the trained multi-task model;
[0010] Step 3: Use visual inspection equipment to collect real-time straw images and strawless images during the production process. Input the straw images and current production control parameter data into the trained multi-task model for processing to obtain real-time defect detection results and straw automatic control parameters.
[0011] Step 4: Transmit the automatic control parameters of the straws to the automatic straw production equipment through the control circuit, and adjust the production parameters in real time;
[0012] Step 5: Continuously collect straw images and production control parameter data during the production process, and automatically iterate and optimize the multi-task model.
[0013] Furthermore, in step 1, several industrial cameras and their matching backlight sources are used to surround and collect images of the extruded straws. While collecting straw images, the production control parameters that need to be recorded include: the real-time operating temperature of the straw production equipment, the extrusion speed, and the current continuous working time.
[0014] Furthermore, in step 1, the acquired images undergo multiple annotation, including:
[0015] Classify and label straw types: that is, label each type of straw image with a different category label compared to images without straws;
[0016] Segment and label the pixels in the eyedropper region: that is, label the eyedropper pixels and background pixels in the image as body labels and background labels, respectively;
[0017] Locate and classify various defects in straws: use rectangular annotation boxes to annotate each defect in the straw sample, and indicate the type of defect;
[0018] Each defect is segmented and labeled with pixels: In each of the above rectangular label boxes containing defects, the pixels of each defect are labeled as defect pixels, and the remaining pixels are labeled as background pixels.
[0019] Furthermore, in step 2, the defect detection and automatic control multi-task model consists of the following parts: image feature extraction network, straw quality feature extraction network, straw classification task head, straw region segmentation task head, defect localization and classification task head, defect segmentation task head, and control parameter optimization task head.
[0020] Furthermore, the image feature extraction network adopts the Feature Pyramid Network (FPN); the straw quality feature extraction network adopts a temporal convolutional neural network that incorporates a Long Short-Term Memory (LSTM) module.
[0021] Furthermore, the straw region segmentation task head, defect localization and classification task head, and defect segmentation task head of the defect detection and automatic control multi-task model have a parameter weight self-selection mechanism; the straw region segmentation task head, defect localization and classification task head, and control parameter optimization task head correspond to each straw type, and save the training results as independent parameter weights; the defect segmentation task head corresponds to each defect type, and saves the training results as independent parameter weights.
[0022] Furthermore, the independent loss functions of the straw classification task head, straw region segmentation task head, defect localization and classification task head, and defect segmentation task head include the cross-entropy function, and the independent loss function of the control parameter optimization task head includes the mean squared error (MSE) loss function. After multiplying the independent loss functions of each task head with different weight parameters, the total loss function of the defect detection and automatic control multi-task model is obtained by summing them.
[0023] Furthermore, in step 3, a visual inspection device is used to collect real-time images of the straw and images without the straw during the production process, including: using multiple industrial cameras and their matching backlight sources, arranged sequentially around the extruded straw, and collecting straw images at a fixed frame rate.
[0024] Furthermore, in step 3, the straw image and continuous running time parameters are input into the trained multi-task model for processing to obtain real-time defect detection results and straw automatic control parameters. The specific process includes:
[0025] C1. Input the real-time captured straw images into the image feature extraction network, and output multi-layer image features through the feature pyramid. ,in The image features of the i-th layer;
[0026] C2. The top-level image features The straw classification task header is input through a fully connected layer to obtain the straw classification results;
[0027] C3. Low-level image features Perform one convolution. Perform one convolution and one 2x upsampling, add the two feature maps obtained from the above operations, and then perform a 4x upsampling operation to obtain the straw region segmentation feature map; input the straw region segmentation feature map into the straw region segmentation task head, and call the corresponding straw region segmentation task parameter weights according to the straw classification result obtained from C2 to obtain the straw pixel region segmentation result;
[0028] C4. Set the eyedropper pixel to 1 and the background pixel to 0 in the input image to obtain the eyedropper region mask; after upsampling the eyedropper region mask according to the image dimensions, perform a dot product with the image features of each layer to eliminate the interference of the background region features, and obtain the image features of the i-th layer eyedropper region. :
[0029]
[0030] in, This is a mask for the eyedropper region after upsampling according to image feature levels;
[0031] C5. Image features of the straw region Input the defect localization and classification task header, call the corresponding defect localization and classification task parameter weights according to the eyedropper classification result, and output the defect localization box and defect classification result for each defect in the image;
[0032] C6. For each defect localization box output by the defect localization and classification task header, generate a defect soft attention mask based on the two-dimensional normal distribution function. :
[0033]
[0034] in, For pixel coordinates, The mean of a two-dimensional normal distribution is given by , where Let the coordinates be the center coordinates of the i-th defect location box. Let be a two-dimensional normal distribution covariance matrix, where and Let be the height and width of the i-th defect localization box, respectively; normalize the amplitudes of all defect soft attention masks and merge them into a defect mask image;
[0035] C7. Image features other than the top and bottom layers. conduct The second convolution and 2x upsampling operation are applied to the lowest-level image features. Perform a convolution operation, sum all the feature maps obtained from each operation, and then perform a 4x upsampling operation to obtain a segmentation feature map with the same pixel resolution as the input image. Multiply the segmentation feature map with the defect mask image obtained in the previous step, and input the resulting defect segmentation feature image into the defect segmentation task head. According to the defect type, call the corresponding defect segmentation parameter weights to obtain the defect pixel segmentation result.
[0036] C8. Based on the defect pixel segmentation results, obtain the type and pixel size of each defect; merge the straw type, defect type and quantity, and defect pixel size output from steps C2, C5, and C7 into real-time straw quality detection results; merge the current working temperature, extrusion speed, and continuous running time of the equipment read from the automatic production equipment into real-time production parameters; input the current straw quality detection results and production parameters into the straw quality feature extraction network, and extract the current production quality features through a convolutional neural network;
[0037] C9. Input the current production quality characteristics into the LSTM network, combine them with the past production quality characteristics recorded by the Long Short-Term Memory module to obtain the time-series quality feature vector; input the time-series quality feature vector into the control parameter optimization task head, call the corresponding control parameter weights according to the straw classification results, and output the production parameter optimization results.
[0038] Furthermore, in step 5, the multi-task model is automatically iterated and optimized, including: recording the defect occurrence rate and corresponding production control parameters in the production process through detection and control programs, using the recorded data as new learning samples, iteratively training the automatic control module in the multi-task model, optimizing its output control parameters, and reducing the probability of defect occurrence.
[0039] The present invention has the following beneficial effects:
[0040] 1. A multi-task learning model containing multiple sub-task modules is used to detect defects in straw images. This solves the problems of poor model transferability, cumbersome model transformation, and inability to accurately screen defect size and type in traditional single-task deep learning classification or localization models, and effectively improves the versatility of the detection model.
[0041] 2. Multiple defect detection task modules use the same network to extract image features. Without incurring additional hardware costs, the overall loss function is determined by weighting. By leveraging the correlation between multiple tasks and the common features between different types of eyedroppers, the detection accuracy of the model is effectively improved and the probability of false detection is reduced.
[0042] 3. By linking defect detection with automatic control optimization, high-order features of real-time product defects and production status parameters are extracted using deep neural networks. Combined with LSTM networks, the current production parameters are dynamically optimized by memorizing and analyzing long-term and short-term defect features. This solves the problems of slow response of manual parameter adjustment, heavy reliance on subjective experience, and high labor costs.
[0043] 4. During the production process, continuously collect defect information corresponding to the parameters, perform semi-supervised iterative optimization on the model, continuously improve the model's defect detection and parameter adjustment capabilities, improve product quality stability, reduce the number of defects, reduce production costs, and avoid waste. Attached Figure Description
[0044] Figure 1 This is a flowchart of a straw defect detection and automatic control method for production parameters according to the present invention.
[0045] Figure 2 This invention describes the processing flow of the multi-task learning model, including information transfer between various internal networks and subtask heads. Detailed Implementation
[0046] The present invention will now be described in further detail with reference to the accompanying drawings and specific preferred embodiments.
[0047] In the description of this invention, it should be understood that the terms "left side," "right side," "upper part," "lower part," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. "First," "second," etc., do not indicate the importance of the components, and therefore should not be construed as a limitation of this invention. The specific dimensions used in this embodiment are only for illustrating the technical solution and do not limit the scope of protection of this invention.
[0048] like Figure 1As shown, a method for detecting straw defects and automatically controlling production parameters includes the following steps:
[0049] Step 1: Manually adjust various production control parameters during the production process, while simultaneously acquiring straw images and recording the production control parameter data at the time of image acquisition. Perform multiple annotations on the acquired straw images to establish a composite sample set containing straw image samples and production parameters.
[0050] The production process should include all possible types of straws, and the production duration for each type of straw should be no less than 8 hours. During the production process, various production parameters should be adjusted based on human experience to simulate various situations in straw production as much as possible.
[0051] In a specific embodiment of this invention, an automatic PLA straw production equipment of a certain model is used for straw image acquisition and production control parameter data recording. During the acquisition and recording period, the equipment continuously produces white single-hole PLA straight straws and white three-hole PLA stirring straws, each type of straw being produced for 10 hours. During production, three industrial cameras and their matching backlight sources are arranged in a 120° angle around the extruded straws, continuously acquiring straw images at a frame rate of 15 FPS. Simultaneously with image acquisition, the production control parameter data at the time of image acquisition is recorded by the automatic control equipment, including: real-time operating temperature, extrusion speed, and current continuous working time. After completing the above production recording, 1000 images without straws are acquired for classification training.
[0052] After image acquisition, the acquired straw images were manually annotated in multiple ways, including: classifying and labeling straw types: different category labels were assigned to each type of straw image and the image without straws; segmenting and labeling straw region pixels: straw pixels and background pixels in the image were labeled as body labels and background labels, respectively; locating and classifying various defects in the straws: each defect in the defective straw sample was labeled using a rectangular annotation box, and the type of defect was noted; and segmenting and labeling each defect: within the rectangular annotation box of each defect, the pixels of each defect were labeled as defect pixels, and the remaining pixels were labeled as background pixels.
[0053] Step 2: Input the composite sample set into the defect detection and automatic control multi-task model for training to obtain the trained multi-task model.
[0054] like Figure 2As shown, in a specific embodiment of the present invention, the defect detection and automatic control multi-task model consists of the following parts: an image feature extraction network, a straw quality feature extraction network, a straw classification task head, a straw region segmentation task head, a defect localization and classification task head, a defect segmentation task head, and a control parameter optimization task head. The image feature extraction network employs a Feature Pyramid Network (FPN); the straw quality feature extraction network employs a temporal convolutional neural network incorporating a Long Short-Term Memory (LSTM) module.
[0055] The aforementioned straw region segmentation task head, defect localization and classification task head, and defect segmentation task head all feature a parameter weight self-selection mechanism. The straw region segmentation task head, defect localization and classification task head, and control parameter optimization task head each correspond to a different straw type, saving the training results as independent parameter weights; the defect segmentation task head corresponds to a different defect type, saving the training results as independent parameter weights.
[0056] The independent loss functions for the straw classification task head, straw region segmentation task head, defect localization and classification task head, and defect segmentation task head include the cross-entropy function, while the independent loss function for the control parameter optimization task head includes the mean squared error (MSE) loss function. The independent loss functions of each task head are multiplied by different weight parameters and then summed to obtain the total loss function of the multi-task model for defect detection and automatic control.
[0057] Step 3: Use visual inspection equipment to collect real-time images of straws and images without straws during the production process. Input the straw images and current production control parameter data into the trained multi-task model for processing to obtain real-time defect detection results and straw automatic control parameters.
[0058] In a specific embodiment of the present invention, the method of using a visual inspection device to acquire real-time straw images and strawless images during the production process includes: using multiple industrial cameras and their matching backlight sources, arranged sequentially around the extruded straw, and acquiring straw images at a fixed frame rate.
[0059] like Figure 2 As shown, the process of inputting the straw image and continuous running time parameters into the trained multi-task model for processing to obtain real-time defect detection results and straw automatic control parameters includes:
[0060] C1. Input the real-time captured straw images into the image feature extraction network, and output multi-layer image features through the feature pyramid. ,in The image features of the i-th layer;
[0061] C2. The top-level image features By inputting the straw classification task header through a fully connected layer, the straw classification result is obtained, that is, the straw type is obtained;
[0062] C3. Low-level image features Perform one convolution. Perform one convolution and one 2x upsampling, add the two feature maps obtained from the above operations, and then perform a 4x upsampling operation to obtain the straw region segmentation feature map. Input the straw region segmentation feature map into the straw region segmentation task head, and call the corresponding straw region segmentation task parameter weights according to the straw classification result obtained from C2 to obtain the straw pixel region segmentation result;
[0063] C4. Set the eyedropper pixel to 1 and the background pixel to 0 in the input image to obtain the eyedropper region mask; after upsampling the eyedropper region mask according to the image dimensions, perform a dot product with the image features of each layer to eliminate the interference of the background region features, and obtain the image features of the i-th layer eyedropper region. :
[0064]
[0065] in, This is a mask for the eyedropper region after upsampling according to image feature levels;
[0066] C5. Image features of the straw region Input the defect localization and classification task header, call the corresponding defect localization and classification task parameter weights according to the eyedropper classification result, and output the defect localization box and defect classification result for each defect in the image, thus obtaining the defect type and quantity.
[0067] C6. For each defect localization box output by the defect localization and classification task header, generate a defect soft attention mask based on the two-dimensional normal distribution function. :
[0068]
[0069] in, For pixel coordinates, The mean of a two-dimensional normal distribution is given by , where Let the coordinates be the center coordinates of the i-th defect location box. Let be a two-dimensional normal distribution covariance matrix, where and Here, represents the height and width of the i-th defect localization box, respectively. The amplitudes of all defect soft attention masks are normalized and merged into a single defect mask image.
[0070] C7. Image features other than the top and bottom layers. conduct The second convolution and 2x upsampling operation are applied to the lowest-level image features. Perform a convolution operation, summing all the feature maps obtained from each operation, and then perform a 4x upsampling operation to obtain a segmentation feature map with the same pixel resolution as the input image. Multiply the segmentation feature map with the defect mask image obtained in the previous step, and input the resulting defect segmentation feature image into the defect segmentation task header. Based on the defect type, call the corresponding defect segmentation parameter weights to obtain the defect pixel segmentation result, i.e., the defect pixel size;
[0071] C8. Based on the defect pixel segmentation results, obtain the type and pixel size of each defect. Combine the straw type, defect type and quantity, and defect pixel size (including defect-free results) output from steps C2, C5, and C7 into a real-time straw quality inspection result. Combine the current operating temperature, extrusion speed, and continuous operating time of the equipment read from the automated production equipment into real-time production parameters. Input the current straw quality inspection result and production parameters into a straw quality feature extraction network, and extract the current production quality features through a convolutional neural network.
[0072] C9. Input the current production quality characteristics into the LSTM network, and combine them with the past production quality characteristics recorded by the Long Short-Term Memory module to obtain the temporal quality feature vector. Input the temporal quality feature vector into the control parameter optimization task head, call the corresponding control parameter weights according to the straw classification results, and output the production parameter optimization results.
[0073] Step 4: Transmit the automatic control parameters of the straws to the automatic straw production equipment through the control circuit, and adjust the production parameters in real time.
[0074] Step 5: Continuously collect straw images and production control parameter data during the production process, and automatically iterate and optimize the multi-task model.
[0075] In a specific embodiment of the present invention, the automatic iteration and optimization update of the multi-task model includes: recording the defect occurrence rate and its corresponding production control parameters in the production process through a detection and control program, using the recorded data as new learning samples, iteratively training the automatic control module in the multi-task model, optimizing its output control parameters, and reducing the probability of defect occurrence.
[0076] This invention employs a multi-task learning model containing various sub-task modules for defect detection in straw images. This solves the problems of poor model transferability, cumbersome model transformation, and inability to accurately filter defect size and type inherent in traditional single-task deep learning classification or localization models, effectively improving the versatility of the detection model. Because the multiple defect detection task modules used in this invention utilize image features extracted by the same network, the overall loss function is determined through weighting without incurring additional hardware costs. By leveraging the correlation between multiple tasks and the common features among different types of straws, the detection accuracy of the model is effectively improved, and the probability of false detections is reduced.
[0077] Building upon optimized defect detection performance, this invention addresses the problems of current manual adjustment of straw production parameters by linking defect detection with automatic control optimization. It utilizes a deep neural network to extract high-order features of real-time product defects and production status parameters, combined with an LSTM network. By memorizing and analyzing long- and short-term defect features, it dynamically optimizes current production parameters, solving problems such as slow response times, heavy reliance on subjective experience, and high labor costs associated with manual parameter adjustment. Simultaneously, it continuously collects defect information corresponding to the parameters during production, performing semi-supervised iterative optimization on the model to continuously improve its defect detection and parameter adjustment capabilities, thereby enhancing product quality stability, reducing the number of defects, lowering production costs, and avoiding waste.
[0078] The preferred embodiments of the present invention have been described in detail above. However, the present invention is not limited to the specific details in the above embodiments. Within the scope of the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all fall within the protection scope of the present invention.
Claims
1. A method for detecting straw defects and automatically controlling production parameters, characterized in that: Includes the following steps: Step 1: Manually adjust various production control parameters during the production process, while simultaneously acquiring straw images and recording the production control parameter data at the time of image acquisition. Perform multiple annotations on the acquired straw images to establish a composite sample set containing straw image samples and production parameters. Step 2: Input the composite sample set into the defect detection and automatic control multi-task model for training to obtain the trained multi-task model; Step 3: Use visual inspection equipment to collect real-time straw images and strawless images during the production process. Input the straw images and current production control parameter data into the trained multi-task model for processing to obtain real-time defect detection results and straw automatic control parameters. Step 4: Transmit the automatic control parameters of the straws to the automatic straw production equipment through the control circuit, and adjust the production parameters in real time; Step 5: Continuously collect straw images and production control parameter data during the production process, and automatically iterate and optimize the multi-task model.
2. The method for detecting straw defects and automatically controlling production parameters according to claim 1, characterized in that: In step 1, several industrial cameras and their matching backlight sources are used to surround and collect images of the extruded straws. While collecting straw images, the production control parameters that need to be recorded include: the real-time operating temperature of the straw production equipment, the extrusion speed, and the current continuous working time.
3. The method for detecting straw defects and automatically controlling production parameters according to claim 1, characterized in that: In step 1, the acquired images undergo multiple annotation, including: Classify and label straw types: that is, label each type of straw image with a different category label compared to images without straws; Segment and label the pixels in the eyedropper region: that is, label the eyedropper pixels and background pixels in the image as body labels and background labels, respectively; Locate and classify various defects in straws: use rectangular annotation boxes to annotate each defect in the straw sample, and indicate the type of defect; Each defect is segmented and labeled with pixels: In each of the above rectangular label boxes containing defects, the pixels of each defect are labeled as defect pixels, and the remaining pixels are labeled as background pixels.
4. The method for detecting straw defects and automatically controlling production parameters according to claim 1, characterized in that: In step 2, the multi-task model for defect detection and automatic control consists of the following parts: image feature extraction network, straw quality feature extraction network, straw classification task head, straw region segmentation task head, defect localization and classification task head, defect segmentation task head, and control parameter optimization task head.
5. The method for detecting straw defects and automatically controlling production parameters according to claim 4, characterized in that: The image feature extraction network uses a feature pyramid network (FPN); the straw quality feature extraction network uses a temporal convolutional neural network that incorporates a long short-term memory module (LSTM).
6. The method for detecting straw defects and automatically controlling production parameters according to claim 4, characterized in that: The straw region segmentation task head, defect localization and classification task head, and defect segmentation task head of the multi-task model for defect detection and automatic control have a parameter weight self-selection mechanism; the straw region segmentation task head, defect localization and classification task head, and control parameter optimization task head correspond to each straw type, and the training results are saved as independent parameter weights. Each defect segmentation task header corresponds to a defect type, and the training results are saved as independent parameter weights.
7. The method for detecting straw defects and automatically controlling production parameters according to claim 4, characterized in that: The independent loss functions for the straw classification task head, straw region segmentation task head, defect localization and classification task head, and defect segmentation task head include the cross-entropy function. The independent loss function for the control parameter optimization task head includes the mean squared error (MSE) loss function. The independent loss functions of each task head are multiplied by different weight parameters and then summed to obtain the total loss function of the defect detection and automatic control multi-task model.
8. The method for detecting straw defects and automatically controlling production parameters according to claim 1, characterized in that: In step 3, a visual inspection device is used to collect real-time images of straws and images without straws during the production process. This includes using multiple industrial cameras and their matching backlight sources, which are arranged sequentially around the extruded straws, and collecting straw images at a fixed frame rate.
9. The method for detecting straw defects and automatically controlling production parameters according to claim 1, characterized in that: In step 3, the straw image and continuous running time parameters are input into the trained multi-task model for processing to obtain real-time defect detection results and straw automatic control parameters. The specific process includes: C1. Input the real-time captured straw images into the image feature extraction network, and output multi-layer image features through the feature pyramid. ,in The image features of the i-th layer; C2. The top-level image features The straw classification task header is input through a fully connected layer to obtain the straw classification results; C3. Low-level image features Perform one convolution. Perform one convolution and one 2x upsampling, add the two feature maps obtained from the above operations, and then perform a 4x upsampling operation to obtain the straw region segmentation feature map; input the straw region segmentation feature map into the straw region segmentation task head, and call the corresponding straw region segmentation task parameter weights according to the straw classification result obtained from C2 to obtain the straw pixel region segmentation result; C4. Set the eyedropper pixel to 1 and the background pixel to 0 in the input image to obtain the eyedropper region mask; after upsampling the eyedropper region mask according to the image dimensions, perform a dot product with the image features of each layer to eliminate the interference of the background region features, and obtain the image features of the i-th layer eyedropper region. : ； in, This is a mask for the eyedropper region after upsampling according to image feature levels; C5. Image features of the straw region Input the defect localization and classification task header, call the corresponding defect localization and classification task parameter weights according to the eyedropper classification result, and output the defect localization box and defect classification result for each defect in the image; C6. For each defect localization box output by the defect localization and classification task header, generate a defect soft attention mask based on the two-dimensional normal distribution function. : ； in, For pixel coordinates, The mean of a two-dimensional normal distribution is given by , where Let the coordinates be the center coordinates of the i-th defect location box. Let be a two-dimensional normal distribution covariance matrix, where and Let be the height and width of the i-th defect localization box, respectively; normalize the amplitudes of all defect soft attention masks and merge them into a defect mask image; C7. Image features other than the top and bottom layers. conduct The second convolution and 2x upsampling operation are applied to the lowest-level image features. Perform a convolution operation, sum all the feature maps obtained from each operation, and then perform a 4x upsampling operation to obtain a segmentation feature map with the same pixel resolution as the input image. Multiply the segmentation feature map with the defect mask image obtained in the previous step, and input the resulting defect segmentation feature image into the defect segmentation task head. According to the defect type, call the corresponding defect segmentation parameter weights to obtain the defect pixel segmentation result. C8. Based on the defect pixel segmentation results, obtain the type and pixel size of each defect; merge the straw type, defect type and quantity, and defect pixel size output from steps C2, C5, and C7 into real-time straw quality detection results; merge the current working temperature, extrusion speed, and continuous running time of the equipment read from the automatic production equipment into real-time production parameters; input the current straw quality detection results and production parameters into the straw quality feature extraction network, and extract the current production quality features through a convolutional neural network; C9. Input the current production quality characteristics into the LSTM network, combine them with the past production quality characteristics recorded by the Long Short-Term Memory module to obtain the time-series quality feature vector; input the time-series quality feature vector into the control parameter optimization task head, call the corresponding control parameter weights according to the straw classification results, and output the production parameter optimization results.
10. The method for detecting straw defects and automatically controlling production parameters according to claim 1, characterized in that: In step 5, the multi-task model is automatically iterated and optimized, including: recording the defect occurrence rate and corresponding production control parameters in the production process through detection and control programs, using the recorded data as new learning samples, iteratively training the automatic control module in the multi-task model, optimizing its output control parameters, and reducing the probability of defect occurrence.