Fiber placement defect detection method and system based on direction-width prior enhancement and deep learning

Abstract

The invention discloses a fiber placement defect detection method and system based on direction-width prior enhancement and deep learning, and the method comprises the steps: embedding layering direction-width joint prior in a visible light architecture, employing direction filtering to suppress old layer texture interference, employing width prior to eliminate cross textures, and employing difference and top cap operation to enhance the defect contrast; an efficient up-sampling module is introduced to replace YOLOv8 common up-sampling, and parameter-quantity-free detail recovery is achieved; a lightweight efficient local attention mechanism is inserted, features are dynamically re-calibrated in a 3 * 3 neighborhood, and high-reflection noisy points are inhibited; and the Qt interface outputs defects in real time and performs closed-loop control. According to the method, accurate defect detection under a fiber placement scene with low contrast, strong cross texture interference and high light reflection is realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image recognition and automatic composite material laying detection, and particularly relates to a filament laying defect detection method and system based on direction-width prior enhancement and deep learning. BACKGROUND

[0002] Automatic filament laying technology is based on carbon fiber reinforced resin matrix composite material, and realizes the functions of high-efficiency forming, high material utilization rate and continuous laying of complex curved surfaces in a numerical control multi-filament bundle parallel laying manner. However, due to reasons such as tension fluctuation and path deviation, surface defects such as gap, lap, wrinkle, foreign matter, filament breakage or twisting are prone to occur in the filament laying process, which weakens the interlayer strength and shortens the fatigue life, and thus must be detected and repaired online.

[0003] However, the following problems exist in the detection of automatic filament laying defects: the high light reflection on the surface of black filaments leads to image gray scale dynamic range compression and contrast reduction; the inherent texture of the filaments presents the characteristics of multiple directions and high density, and is easily identified as a pseudo defect; the interlayer texture coupling is caused by the cross-laying angle, and background interference is easily formed; in addition, the above factors jointly cause the increase of defect omission rate and false alarm rate, and the decrease of detection reliability.

[0004] In view of the above problems, the existing automatic filament laying defect detection technology solves the problems in the following four ways: manual visual inspection, laser scanning, infrared thermal imaging and visible light imaging. However, manual visual inspection is limited by spatial resolution and visual fatigue, and has low detection efficiency and consistency; although the laser scanning method can provide micron-level three-dimensional topography and is not sensitive to the black high-reflective surface of the filaments, such as using a line laser to calibrate the laying boundary, it can effectively eliminate the black light reflection, but the point cloud data of this kind of method is huge, and the real-time performance is insufficient; the infrared thermal imaging realizes defect visualization by passively capturing the global temperature difference, and although it is not sensitive to the color and light reflection of the filaments, such as fusing infrared and visible light to suppress texture interference, this kind of method needs to strictly shield environmental heat sources, and has complex equipment and high maintenance cost; the visible light imaging has the advantages of low cost and easy maintenance, but it relies on external lighting, and has high omission and misdiagnosis rates in the scene of black surface, high light reflection, low contrast and complex texture, such as using a CCD-edge geometric feature to improve the detection speed, but the texture lacks, leading to difficult classification, such as using a shallow RBF network for classification, but the generalization is insufficient. SUMMARY

[0005] The application provides a filament laying defect detection method and system based on direction-width prior enhancement and deep learning, and aims to solve the problems of low contrast of filament surface, strong cross-texture interference, high light reflection and insufficient defect detection precision in the automatic filament laying scene.

[0006] Technical solution: The application provides a filament laying defect detection method based on direction-width prior enhancement and deep learning, comprising:

[0007] Collecting a filament laying image;

[0008] Pretreating the filament laying image, including direction prior, width prior, dark area difference and bright area top hat;

[0009] Replacing the basic up-sampling module in the original YOLOv8 network with an efficient up-sampling module, and adding an efficient local attention mechanism to obtain an improved YOLOv8 network, and inputting the pretreated filament laying image into the improved YOLOv8 network for defect detection; the efficient up-sampling module enlarges the size of the feature map input into the efficient up-sampling module through bilinear interpolation; the efficient local attention mechanism is used to enhance the features of the feature map input into the efficient local attention mechanism;

[0010] Visualizing and outputting the defect detection result and performing closed-loop control.

[0011] Further, the pretreatment of the filament laying image comprises:

[0012] Performing two-dimensional Fourier transform and centering on the filament laying image, after masking the low frequency of the center circle domain, integrating to obtain the angle energy distribution, taking the peak angle of the first two layers to construct the direction filter, filtering the features according to the filtering direction, and obtaining the direction prior image;

[0013] A rectangular structure element is constructed to perform an opening operation on the direction prior image to obtain a filtered image; the short axis of the rectangular structure element is equal to the pixel width of the filament bundle in the direction prior image, and the long axis is along the current filament laying direction;

[0014] Performing a dark area difference operation on the filtered image, and simultaneously performing a bright area top hat transformation to obtain the pretreated filament laying image.

[0015] Further, the efficient up-sampling module comprises: an up-sampling unit, a feature enhancement unit, a normalization and activation unit, and a channel adjustment unit; wherein the up-sampling unit expands the scale of the feature map input into the efficient up-sampling module by 2 times in a bilinear interpolation manner, obtains an expanded domain, and aligns the spatial resolution; the feature enhancement unit adopts a 3x3 depth separable convolution to extract local spatial features in the expanded domain; the normalization and activation unit sequentially performs batch normalization and ReLU activation function on the feature map processed by the feature enhancement unit, and normalizes the distribution of the extracted local spatial features; the channel adjustment unit unifies the channel dimension through 1x1 point-by-point convolution.

[0016] Further, the high-efficiency local attention mechanism comprises a strip pooling unit, a convolution unit, a normalization and activation unit, and a weight fusion unit; the strip pooling unit performs average pooling on feature maps input into the high-efficiency local attention mechanism along horizontal and vertical directions respectively, extracts horizontal feature vectors and vertical feature vectors with position coding characteristics, the convolution unit performs one-dimensional convolution on the horizontal feature vectors and the vertical feature vectors, expands a local receptive field and enhances direction interaction, the normalization and activation unit adopts group normalization and a Sigmoid activation function to generate horizontal attention weights and vertical attention weights for feature maps processed by the convolution unit, and the weight fusion unit point-by-point multiplies two groups of weights to obtain comprehensive spatial attention weights, and performs element-by-element multiplication on the comprehensive spatial attention weights and the feature maps input into the high-efficiency local attention mechanism to complete feature enhancement.

[0017] Further, the visualization module is configured to visualize and output the defect detection result and perform closed-loop control.

[0018] The application provides a filament laying defect detection system based on direction-width prior enhancement and deep learning, comprising:

[0019] The acquisition module is configured to acquire the filament laying image.

[0020] The preprocessing module is configured to preprocess the filament laying image, including direction prior, width prior, dark area difference and bright area top hat.

[0021] The model building module is configured to replace a basic up-sampling module in an original YOLOv8 network with a high-efficiency up-sampling module, add a high-efficiency local attention mechanism, obtain an improved YOLOv8 network, and input the preprocessed filament laying image into the improved YOLOv8 network to perform defect detection; the high-efficiency up-sampling module enlarges the size of a feature map input into the high-efficiency up-sampling module by means of bilinear interpolation; and the high-efficiency local attention mechanism is used to perform feature enhancement on the feature map input into the high-efficiency local attention mechanism.

[0022] The visualization module is configured to visualize and output the defect detection result and perform closed-loop control.

[0023] Further, in the preprocessing module, the preprocessing of the filament laying image comprises:

[0024] The filament laying image is subjected to two-dimensional Fourier transform and centering, the low frequency in a center circle domain is masked, the angle energy distribution is obtained by integration, the first two peak angle values are taken to construct a direction filter, the features are filtered according to the filtering direction, and a direction prior image is obtained.

[0025] The rectangular structure element is used to open the operation of the direction prior image to obtain a filtered image; the short axis of the rectangular structure element is equal to the pixel width of the tows in the direction prior image, and the long axis is along the current tows laying direction;

[0026] The dark area difference operation is performed on the filtered image, and the bright area top hat transformation is synchronously performed to obtain a preprocessed tows laying image.

[0027] Further, in the model building module, the high-efficiency up-sampling module includes: an up-sampling unit, a feature enhancement unit, a normalization and activation unit, and a channel adjustment unit; the up-sampling unit expands the scale of the feature map input into the high-efficiency up-sampling module by 2 times in a bilinear interpolation manner to obtain an expanded domain and align the spatial resolution; the feature enhancement unit extracts local spatial features in the expanded domain by using a 3*3 depth separable convolution; the normalization and activation unit sequentially performs batch normalization and ReLU activation function on the feature map processed by the feature enhancement unit to normalize the distribution of the extracted local spatial features; and the channel adjustment unit unifies the channel dimension by using 1*1 point-by-point convolution.

[0028] Further, in the model building module, the high-efficiency local attention mechanism includes: a strip pooling unit, a convolution unit, a normalization and activation unit, and a weight fusion unit; the strip pooling unit performs average pooling on the feature map input into the high-efficiency local attention mechanism along the horizontal and vertical directions to extract horizontal feature vectors and vertical feature vectors with position coding characteristics; the convolution unit performs one-dimensional convolution on the horizontal feature vectors and the vertical feature vectors to expand the local receptive field and enhance the interaction between directions; the normalization and activation unit uses group normalization and Sigmoid activation function to generate horizontal attention weights and vertical attention weights for the feature map processed by the convolution unit; and the weight fusion unit point-by-point multiplies the two groups of weights to obtain comprehensive spatial attention weights, and performs element-by-element multiplication on the feature map input into the high-efficiency local attention mechanism according to the channel to complete feature enhancement.

[0029] Further, in the visualization module, the visualization outputs the defect detection result and performs closed-loop control, including: building a graphical man-machine interface based on a Qt framework, synchronously superimposedly displaying a tows laying image, a defect mask obtained by YOLOv8 defect detection, a minimum bounding rectangle of the defect, recording the defect parameters, and stopping the tows laying head through online feedback control.

[0030] Beneficial effects: the present application provides a silk laying defect detection method and system based on direction-width prior enhancement and deep learning, compared with prior art, the present application embeds "paving layer direction-width" joint prior in the visible light architecture: the direction filter suppresses the old layer texture interference, the width prior eliminates the cross texture, the difference and top hat operation enhances the defect contrast; an efficient up-sampling module is introduced to replace the YOLOv8 ordinary up-sampling, realizing parameter-free detail recovery; a lightweight efficient local attention mechanism is inserted, dynamically recalibrating features in 3*3 neighborhood, suppressing high-reflective noise points; Qt interface outputs defects in real time and realizes closed-loop control. The method realizes accurate defect detection in the silk laying scene with low contrast, strong cross texture interference and high reflectivity. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 The present application provides a silk laying defect detection method and system based on direction-width prior enhancement and deep learning, compared with prior art, the present application embeds "paving layer direction-width" joint prior in the visible light architecture: the direction filter suppresses the old layer texture interference, the width prior eliminates the cross texture, the difference and top hat operation enhances the defect contrast; an efficient up-sampling module is introduced to replace the YOLOv8 ordinary up-sampling, realizing parameter-free detail recovery; a lightweight efficient local attention mechanism is inserted, dynamically recalibrating features in 3*3 neighborhood, suppressing high-reflective noise points; Qt interface outputs defects in real time and realizes closed-loop control. The method realizes accurate defect detection in the silk laying scene with low contrast, strong cross texture interference and high reflectivity.

[0032] Figure 2 The present application provides a silk laying defect detection method and system based on direction-width prior enhancement and deep learning, compared with prior art, the present application embeds "paving layer direction-width" joint prior in the visible light architecture: the direction filter suppresses the old layer texture interference, the width prior eliminates the cross texture, the difference and top hat operation enhances the defect contrast; an efficient up-sampling module is introduced to replace the YOLOv8 ordinary up-sampling, realizing parameter-free detail recovery; a lightweight efficient local attention mechanism is inserted, dynamically recalibrating features in 3*3 neighborhood, suppressing high-reflective noise points; Qt interface outputs defects in real time and realizes closed-loop control. The method realizes accurate defect detection in the silk laying scene with low contrast, strong cross texture interference and high reflectivity.

[0033] Figure 3 The present application provides a silk laying defect detection method and system based on direction-width prior enhancement and deep learning, compared with prior art, the present application embeds "paving layer direction-width" joint prior in the visible light architecture: the direction filter suppresses the old layer texture interference, the width prior eliminates the cross texture, the difference and top hat operation enhances the defect contrast; an efficient up-sampling module is introduced to replace the YOLOv8 ordinary up-sampling, realizing parameter-free detail recovery; a lightweight efficient local attention mechanism is inserted, dynamically recalibrating features in 3*3 neighborhood, suppressing high-reflective noise points; Qt interface outputs defects in real time and realizes closed-loop control. The method realizes accurate defect detection in the silk laying scene with low contrast, strong cross texture interference and high reflectivity.

[0034] Figure 4 The present application provides a silk laying defect detection method and system based on direction-width prior enhancement and deep learning, compared with prior art, the present application embeds "paving layer direction-width" joint prior in the visible light architecture: the direction filter suppresses the old layer texture interference, the width prior eliminates the cross texture, the difference and top hat operation enhances the defect contrast; an efficient up-sampling module is introduced to replace the YOLOv8 ordinary up-sampling, realizing parameter-free detail recovery; a lightweight efficient local attention mechanism is inserted, dynamically recalibrating features in 3*3 neighborhood, suppressing high-reflective noise points; Qt interface outputs defects in real time and realizes closed-loop control. The method realizes accurate defect detection in the silk laying scene with low contrast, strong cross texture interference and high reflectivity.

[0035] Figure 5 The present application provides a silk laying defect detection method and system based on direction-width prior enhancement and deep learning, compared with prior art, the present application embeds "paving layer direction-width" joint prior in the visible light architecture: the direction filter suppresses the old layer texture interference, the width prior eliminates the cross texture, the difference and top hat operation enhances the defect contrast; an efficient up-sampling module is introduced to replace the YOLOv8 ordinary up-sampling, realizing parameter-free detail recovery; a lightweight efficient local attention mechanism is inserted, dynamically recalibrating features in 3*3 neighborhood, suppressing high-reflective noise points; Qt interface outputs defects in real time and realizes closed-loop control. The method realizes accurate defect detection in the silk laying scene with low contrast, strong cross texture interference and high reflectivity.

[0036] Figure 6 The present application provides a silk laying defect detection method and system based on direction-width prior enhancement and deep learning, compared with prior art, the present application embeds "paving layer direction-width" joint prior in the visible light architecture: the direction filter suppresses the old layer texture interference, the width prior eliminates the cross texture, the difference and top hat operation enhances the defect contrast; an efficient up-sampling module is introduced to replace the YOLOv8 ordinary up-sampling, realizing parameter-free detail recovery; a lightweight efficient local attention mechanism is inserted, dynamically recalibrating features in 3*3 neighborhood, suppressing high-reflective noise points; Qt interface outputs defects in real time and realizes closed-loop control. The method realizes accurate defect detection in the silk laying scene with low contrast, strong cross texture interference and high reflectivity. DETAILED DESCRIPTION

[0037] The present application provides a silk laying defect detection method and system based on direction-width prior enhancement and deep learning, compared with prior art, the present application embeds "paving layer direction-width" joint prior in the visible light architecture: the direction filter suppresses the old layer texture interference, the width prior eliminates the cross texture, the difference and top hat operation enhances the defect contrast; an efficient up-sampling module is introduced to replace the YOLOv8 ordinary up-sampling, realizing parameter-free detail recovery; a lightweight efficient local attention mechanism is inserted, dynamically recalibrating features in 3*3 neighborhood, suppressing high-reflective noise points; Qt interface outputs defects in real time and realizes closed-loop control. The method realizes accurate defect detection in the silk laying scene with low contrast, strong cross texture interference and high reflectivity.

[0038] Example one

[0039] As Figure 1 shown, the present application provides a silk laying defect detection method based on direction-width prior enhancement and deep learning, comprising:

[0040] collecting a silk laying image;

[0041] preprocessing the silk laying image, including direction prior, width prior, dark area difference and bright area top hat;

[0042] The basic upsampling module in the original YOLOv8 network is replaced by an efficient upsampling module, and an efficient local attention mechanism is added to obtain an improved YOLOv8 network; the efficient upsampling module enlarges the size of the feature map input into the efficient upsampling module through bilinear interpolation; the efficient local attention mechanism is used to enhance the features of the feature map input into the efficient local attention mechanism;

[0043] The defect detection result is visualized and closed-loop control is performed.

[0044] In the embodiment,

[0045] As shown in Figure 2 , an industrial camera is arranged horizontally and a hollow frame-shaped low-angle ring-shaped light source is arranged, and the two are coaxially fixed to the filament laying head through a lateral support and move integrally with the filament laying head to realize detection while laying filaments without the need to add a scanning shaft; the camera is located above the light source, the lower end surface of the lens thereof is kept at a safe distance of 25-40 mm from the surface of the composite material, the angle between the light emitting surface of the light source and the horizontal plane is 10°-20°, the center transparent window has a light transmittance of ≥90%, and the lower surface of the window is 5-15 mm away from the surface of the composite material to form a low-angle ring-shaped illumination and avoid collision; the light source is hard triggered by the camera exposure signal to realize millisecond-level synchronization of illumination and imaging.

[0046] As shown in Figure 3 , the filament laying image is preprocessed, including:

[0047] The filament laying image is subjected to two-dimensional Fourier transform and centering, the angle energy distribution is obtained by integrating after masking the low frequency of the center circular domain, the first two peak angle values are taken to construct a direction filter, and the direction prior image is obtained by filtering the features according to the filtering direction;

[0048] The direction prior image is subjected to an open operation by constructing a rectangular structure element to obtain a filtered image; the short axis of the rectangular structure element is equal to the pixel width of the filament bundle in the direction prior image, and the long axis is along the current filament laying direction;

[0049] The dark region difference operation is performed on the filtered image, and the bright region top-hat transformation is simultaneously performed to obtain the preprocessed filament laying image.

[0050] Specifically, the collected filament laying image, i.e., the input grayscale image is subjected to two-dimensional discrete Fourier transform to obtain a complex spectrum , and the zero-frequency component is moved to the center of the spectrum by using matrix coordinate reshooting; then a center circular domain mask is constructed to shield the low-frequency fundamental component and retain the high-frequency texture information to obtain a filtered spectrum ; the is converted to polar coordinates , and the range to calculate the energy of each direction , forming an angular energy distribution curve; after smoothing the curve, the peak value is detected, and the maximum peak value corresponding angle is taken as the main direction of the current layer of tows, and the peak value angle that is the second highest and differs from by ≥15° is taken as the direction of the previous layer of tows; finally, a direction filtering mask is constructed , and the response coefficients in the range are set to zero in the gradient amplitude domain to effectively suppress the spectrum energy of the cross texture of the previous layer and obtain a direction prior image, which only retains the features consistent with the direction of the current layer for subsequent defect detection.

[0051] A rectangular structure element is constructed according to the current tow width W, the short axis length of which is equal to W, the long axis length of which is equal to 3W, and the long axis direction is strictly aligned with the design angle of the layer to ensure that the subsequent opening operation only acts on the gap region between the tows, effectively removing dust, while avoiding erosion of the tow body or causing edge blur. The morphological opening operation performs an operation of first erosion and then inflation on the direction prior image; the residual error extraction obtains a potential defect response image D1 by pixel-by-pixel subtraction of the opening operation result from the original image, which shows the dark area or highlight abnormal point removed by the opening operation; at the same time, a binary mask is generated to mark the opening operation removal area, providing accurate spatial basis for subsequent connected domain area screening and defect-noise separation.

[0052] Perform a dark area difference operation on the filtered image, with a threshold of 25-40 gray levels; simultaneously perform a bright area top hat transformation with a threshold of 93-96 gray values, and remove connected domains with an area less than 200 pixels to improve the defect signal-to-noise ratio. The dark area difference step subtracts a fixed bias of 28 gray levels from the potential defect response image pixel by pixel, and obtains a dark line response image D2 through zero truncation, which only retains the dark defect edges with low depth and significant contrast to suppress the background gray level drift. The bright area top hat transformation is performed on D1: a circular structure element with a diameter of 7 pixels is used to perform an opening operation reconstruction on the high-light area with gray levels of 93-96, to extract isolated regions that are brighter than the adjacent background, and generate a bright spot response image D3, to achieve high-light feature enhancement of small protrusions, foreign matter or fiber bundle breaks. Joint mask filtering linearly adds D2 and D3 to obtain a total response image D4, and then removes connected domains with an area less than 200 pixels to ensure that the retained defect regions have spatial continuity. Dynamic range stretching performs a 5%-95% percentile linear mapping on D4 to output an enhanced image, so that dark lines and bright spots occupy the maximum quantization interval at the same time, ensuring that the subsequent deep learning network obtains high-contrast and high-signal-to-noise ratio input data.

[0053] ​The basic upsampling module in the original YOLOv8 network is replaced by an efficient upsampling module, and an efficient local attention mechanism is added to obtain an improved YOLOv8 network, and the preprocessed filament laying image is input into the improved YOLOv8 network for defect detection; the efficient upsampling module enlarges the size of the feature map input into the efficient upsampling module through bilinear interpolation; the efficient local attention mechanism is used to enhance the features of the feature map input into the efficient local attention mechanism.

[0054] Specifically, as shown in the figure, Figure 4 The improved YOLOv8 network adopts a single-stage anchor-free framework. The backbone is composed of Conv convolution units and C2f modules in an alternating and serial manner, and is connected to an SPPF spatial pyramid pooling component at the end for generating feature maps of three different spatial resolutions; the neck rapidly restores details through an efficient upsampling module (EUM) and fuses deep and shallow information through a C2f module to realize bidirectional feature transmission from bottom to top and from top to bottom; the head is used to directly predict the boundary box offset, target confidence and class probability, and the efficient local attention mechanism (ELAM) is embedded before the detection head (detect) to strengthen the response of the defect area and thus optimize the detection effect.

[0055] As shown in the figure, Figure 5 The efficient upsampling module (Efficient Upsampling Module, EUM) includes an upsampling unit (up2x), a feature enhancement unit (DWC3x3), a normalization and activation unit (BN, Relu), and a channel adjustment unit (Conv1x1). The upsampling unit uses bilinear interpolation to expand the scale of the feature map input into the efficient upsampling module by 2 times, obtains an expanded domain, and aligns the spatial resolution; the feature enhancement unit uses a 3x3 depth separable convolution to extract local spatial features in the expanded domain; the normalization and activation unit sequentially performs batch normalization (Batch Normalization) and rectified linear unit (ReLU) activation to realize feature distribution normalization and introduce nonlinear expression capability, ensuring numerical stability and gradient transmission; the channel adjustment unit unifies the channel dimension through 1x1 point-by-point convolution, so that the output is strictly consistent with the network skip connection and the subsequent layer, providing a dimensional basis for deep and shallow feature fusion, strengthening defect local feature expression and suppressing background interference.

[0056] As shown in the figure, Figure 6As shown, C is the channel number of the feature map input into the efficient local attention mechanism, H is the height of the feature map input into the efficient local attention mechanism, and W is the width of the feature map input into the efficient local attention mechanism, and the efficient local attention mechanism comprises a strip pooling unit (X Avg Pool, Y Avg Pool), a convolution unit (Conv1d), a normalization and activation unit (GroupNorm, Sigmoid), and a weight fusion unit; wherein the strip pooling unit performs average pooling along the horizontal and vertical directions respectively to extract bidirectional feature vectors (including horizontal feature vectors and vertical feature vectors) with position coding characteristics; the convolution unit performs one-dimensional convolution on the horizontal feature vectors and the vertical feature vectors to expand the local receptive field and enhance the interaction between directions; the normalization and activation unit adopts group normalization (Group Normalization) and a Sigmoid activation function to generate horizontal attention weights and vertical attention weights for the feature map processed by the convolution unit; and the weight fusion unit point-by-point multiplies the two groups of weights to obtain comprehensive spatial attention weights, and performs element-by-element multiplication on the original feature map input into the efficient local attention mechanism according to the channels to complete feature enhancement.

[0057] The defect detection result is visualized and closed-loop control is performed, including: a graphical man-machine interface is built based on a Qt framework, a laying image, a defect mask obtained by YOLOv8 defect detection, and a minimum bounding rectangle of a defect are synchronously superimposed and displayed, defect parameters are recorded, and an online feedback control is performed to stop a laying head.

[0058] Specifically, the Qt interface integrates a main view rendering module, a defect parameter table module, and a real-time interaction and closed-loop control module to jointly complete graphical presentation of defect information and closed-loop disposal on a production line: the main view superimposes an original laying image, a defect mask, and a minimum bounding rectangle to a QGraphicsScene, supports scaling, highlighting, and color-category binding; the table writes defect numbers, categories, confidence levels, center coordinates, widths, and heights row by row through a QTableWidget and is bidirectionally linked with the main view; and the closed-loop control module encapsulates the minimum bounding rectangle parameters into a JSON frame, periodically sends the JSON frame to a PLC through Ethernet, and triggers the laying head to stop immediately after the PLC parses the JSON frame, thereby completing a detection-control closed loop.

[0059] Embodiment Two

[0060] Based on Embodiment One, as shown in Figure 1 The application also provides a laying defect detection system based on direction-width prior enhancement and deep learning, comprising:

[0061] A collecting module is configured to collect a laying image.

[0062] a preprocessing module, configured to pre-process the filament laying image, including direction prior, width prior, dark region difference, and bright region top hat;

[0063] a model building module, configured to replace a basic up-sampling module in an original YOLOv8 network with an efficient up-sampling module, and add an efficient local attention mechanism to obtain an improved YOLOv8 network, and input the pre-processed filament laying image into the improved YOLOv8 network for defect detection; the efficient up-sampling module enlarges the size of a feature map input into the efficient up-sampling module through bilinear interpolation; the efficient local attention mechanism is used to perform feature enhancement on a feature map input into the efficient local attention mechanism;

[0064] a visualization module, configured to visualize and output the defect detection result and perform closed-loop control.

[0065] In this embodiment,

[0066] As shown in Figure 2 , the acquisition module adopts a horizontal industrial camera and a hollow frame-shaped low-angle ring light source, which are coaxially fixed to the filament laying head through a lateral support and move integrally with the filament laying head, realizing detection while laying the filament without the need to add a scanning shaft; the camera is located above the light source, the lower end surface of the lens thereof maintains a safety distance of 25-40 mm from the surface of the composite material, the light-emitting surface of the light source forms an angle of 10°-20° with the horizontal plane, the central transparent window has a light transmittance of ≥90 %, and the lower surface of the window is 5-15 mm away from the surface of the composite material, forming a low-angle ring illumination and avoiding collision; the light source is hard-triggered by the camera exposure signal, realizing millisecond-level synchronization of illumination and imaging.

[0067] As shown in Figure 3 , the preprocessing module pre-processes the filament laying image, including:

[0068] performing two-dimensional Fourier transform and centering on the filament laying image, after masking the low frequency of the central circular domain, integrating to obtain the angle energy distribution, taking the peak angle of the first two layers to construct the direction filter, filtering the features according to the filtering direction to obtain the direction prior image;

[0069] performing an opening operation on the direction prior image by using a rectangular structure element to obtain a filtered image; the short axis of the rectangular structure element is equal to the pixel width of the filament in the direction prior image, and the long axis is along the current filament laying direction;

[0070] performing a dark region difference operation on the filtered image, and simultaneously performing a bright region top hat transform to obtain the pre-processed filament laying image.

[0071] Specifically, for the collected filament laying image, i.e., the input grayscale image performing two-dimensional discrete Fourier transform to obtain a complex spectrum The zero-frequency component is moved to the center of the spectrum by using matrix coordinate re-beating; then a central circular mask is constructed to shield the low-frequency fundamental frequency component while retaining the mid-to-high frequency texture information, thus obtaining the filtered spectrum. ;Will Transform to polar coordinates ,exist Within the range Integrate over the entire path with step size and ρ, and calculate the energy in each direction. An angular energy distribution curve is formed; after smoothing the curve, the peak value is detected, and the angle corresponding to the maximum peak value is taken. As the current main direction of the plywood bundle, and second highest and with Peak angles differing by ≥15° As the direction of the front layer filament bundle; finally, a directional filter mask is constructed. In the gradient magnitude domain The response coefficients within the interval are set to zero to effectively suppress the spectral energy of the previous layer's cross-texture, obtain the directional prior image, and retain only the features consistent with the current layer's direction for subsequent defect detection.

[0072] Based on the current filament width W, a rectangular structural element is constructed, with its minor axis length equal to W and its major axis length equal to 3W, and the direction of the major axis is at an angle to the layup design. Strict alignment is maintained to ensure that subsequent opening operations only affect the gaps between the filaments, effectively removing dust while avoiding erosion of the filament body or causing edge blurring. Morphological opening operations perform an erosion-dilation operation on the directional prior image; residual extraction obtains the potential defect response map D1 by subtracting the opening operation result from the original image pixel by pixel. This image shows the dark areas or bright anomalies removed by the opening operation; simultaneously, a binary mask is generated to mark the areas removed by the opening operation, providing precise spatial basis for subsequent connected component area screening and defect-noise separation.

[0073] A dark region difference operation is performed on the filtered image, with a threshold of 25-40 gray levels; a bright region top-hat transformation is performed synchronously, with a threshold of 93-96 gray values, and connected domains with an area less than 200 pixels are removed, so as to improve the defect signal-to-noise ratio. The dark region difference step subtracts a fixed bias of 28 gray levels from the potential defect response map pixel by pixel, and a dark line response map D2 is obtained through zero truncation, only retaining dark defects with low depth and significant contrast, so as to suppress the substrate gray level drift. The bright region top-hat transformation is performed on D1: a circular structure element with a diameter of 7 pixels is used to perform an opening operation reconstruction on the gray 93-96 quantile high-light region, to extract isolated regions brighter than the adjacent background, to generate a bright spot response map D3, to realize high-light feature enhancement of small convex, foreign matter or fiber bundle fracture. Joint mask filtering linearly adds D2 and D3 to obtain a total response map D4, and then removes connected domains with an area less than 200 pixels, so as to ensure that the retained defect regions have spatial continuity. Dynamic range stretching performs 5%-95% percentile linear mapping on D4, to output an enhanced map, so that dark lines and bright spots occupy the maximum quantization interval at the same time, to ensure that the subsequent deep learning network obtains high-contrast and high-signal-to-noise ratio input data.

[0074] The model building module replaces the basic upsampling module in the original YOLOv8 network with an efficient upsampling module, and adds an efficient local attention mechanism to obtain an improved YOLOv8 network, and inputs the preprocessed filament laying image into the improved YOLOv8 network for defect detection; the efficient upsampling module enlarges the size of the feature map input into the efficient upsampling module through bilinear interpolation; the efficient local attention mechanism is used to enhance the features of the feature map input into the efficient local attention mechanism.

[0075] Specifically, as shown in Figure 4 The improved YOLOv8 network adopts a single-stage anchor-free framework. The backbone is composed of Conv convolution units and C2f modules in an alternating and serial manner, and is connected with an SPPF spatial pyramid pooling component at the end, which is used to generate feature maps with three different spatial resolutions; the neck part uses an efficient upsampling module (EUM) to quickly restore details, and fuses deep and shallow information through a C2f module to realize bidirectional transmission of features from bottom to top and from top to bottom; the head part is used to directly predict the boundary box offset, target confidence and class probability, and the efficient local attention mechanism (ELAM) is embedded in the detection head (detect) to strengthen the response of the defect area, thereby optimizing the detection effect.

[0076] As shown in Figure 5As shown, the efficient up-sampling module (Efficient Upsampling Module, EUM) includes: an up-sampling unit (up2x), a feature enhancement unit (DWC3x3), a normalization and activation unit (BN, Relu), and a channel adjustment unit (Conv1x1). The up-sampling unit expands the feature map scale input into the efficient up-sampling module by 2 times in a bilinear interpolation manner, obtains an expanded domain, and aligns the spatial resolution. The feature enhancement unit uses a 3x3 depth separable convolution to extract local spatial features in the expanded domain. The normalization and activation unit sequentially performs batch normalization (Batch Normalization) and rectified linear unit (ReLU) activation, normalizes the feature distribution, introduces non-linear expression capability, ensures numerical stability and gradient transmission, and unifies the channel dimension through 1x1 point-by-point convolution, so that the output is strictly consistent with the network jump connection and the subsequent layer, providing a dimensional basis for deep and shallow feature fusion, strengthening the defect local feature expression and suppressing background interference.

[0077] As shown in Figure 6 C is the channel number (Channel) of the feature map input into the efficient local attention mechanism, H is the height (Height) of the feature map input into the efficient local attention mechanism, and W is the width (Width) of the feature map input into the efficient local attention mechanism. The efficient local attention mechanism includes: a strip pooling unit (X Avg Pool, Y Avg Pool), a convolution unit (Conv1d), a normalization and activation unit (GroupNorm, Sigmoid), and a weight fusion unit. The strip pooling unit performs average pooling along the horizontal and vertical directions respectively to extract bidirectional feature vectors (including horizontal feature vectors and vertical feature vectors) with position coding characteristics. The convolution unit performs one-dimensional convolution on the horizontal feature vectors and the vertical feature vectors to expand the local receptive field and enhance the interaction between directions. The normalization and activation unit uses group normalization (Group Normalization) and a Sigmoid activation function to generate horizontal attention weights and vertical attention weights for the feature map processed by the convolution unit. The weight fusion unit point-by-point multiplies the two groups of weights to obtain comprehensive spatial attention weights, and performs element-by-element multiplication with the original feature map input into the efficient local attention mechanism by channel to complete feature enhancement.

[0078] The visualization module visualizes and outputs the defect detection result and performs closed-loop control, including: building a graphical man-machine interface based on a Qt framework, synchronously superimposedly displaying a filament laying image, a defect mask obtained by YOLOv8 defect detection, a minimum bounding rectangle of a defect, recording defect parameters, and stopping a filament laying head through online feedback control.

[0079] Specifically, Qt interface integrates the main view rendering module, defect parameter table module and real-time interaction and closed-loop control module to jointly complete the graphical presentation of defect information and the closed-loop disposal of the production line: the main view superimposes the original fiber placement image, defect mask and minimum bounding rectangle to QGraphicsScene, supports scaling, highlighting and color-category binding; the table writes defect number, category, confidence, center coordinates, width and height through QTableWidget row by row, and is bidirectionally linked with the main view; the closed-loop control module encapsulates the minimum bounding rectangle parameters into a JSON frame, which is periodically sent to the PLC through Ethernet, and the PLC triggers the fiber placement head to stop immediately after parsing, completing the detection-control closed loop.

Claims

1. A method for detecting wire lay defects based on direction-width prior enhancement and deep learning, characterized in that, include: Acquire images of the fiber layup; The wire-laying image is preprocessed, including orientation prior, width prior, dark area difference, and bright area top cap; The improved YOLOv8 network is obtained by replacing the base sampling module in the original YOLOv8 network with an efficient upsampling module and adding an efficient local attention mechanism. The preprocessed wire-laying image is then input into the improved YOLOv8 network for defect detection. The high-efficiency upsampling module enlarges the size of the feature map input to the high-efficiency upsampling module through bilinear interpolation. An efficient local attention mechanism is used to enhance features on the feature map input to the efficient local attention mechanism; Visualize the defect detection results and implement closed-loop control.

2. The method for detecting wire lay defects based on direction-width prior enhancement and deep learning according to claim 1, characterized in that, The preprocessing of the fiber-laying image includes: The filament-laying image is subjected to a two-dimensional Fourier transform and centered. After masking the low frequency of the central circular domain, the angular energy distribution is obtained by integration. The peak angles of the first two layers are used to construct a directional filter. Based on the filtering directional filtering features, the directional prior image is obtained. A rectangular structural element is constructed to perform an opening operation on the orientation prior image to obtain a filtered image; the minor axis of the rectangular structural element is equal to the pixel width of the filament bundle in the orientation prior image, and the major axis is along the current filament laying direction. Perform dark area difference operation on the filtered image and simultaneously perform bright area top cap transformation to obtain the preprocessed wire-laying image.

3. The method for detecting wire lay defects based on direction-width prior enhancement and deep learning according to claim 1, characterized in that, The efficient upsampling module includes: an upsampling unit, a feature enhancement unit, a normalization and activation unit, and a channel adjustment unit. The upsampling unit uses bilinear interpolation to enlarge the feature map input to the efficient upsampling module by a factor of two, obtaining an expanded domain and aligning it with the spatial resolution. The feature enhancement unit uses a 3×3 depthwise separable convolution to extract local spatial features within the expanded domain. The normalization and activation unit sequentially performs batch normalization and ReLU activation on the feature map processed by the feature enhancement unit to normalize the distribution of the extracted local spatial features. The channel adjustment unit unifies the channel dimensions through a 1×1 pointwise convolution.

4. The method for detecting wire lay defects based on direction-width prior enhancement and deep learning according to claim 1, characterized in that, The efficient local attention mechanism includes: a strip pooling unit, a convolution unit, a normalization and activation unit, and a weight fusion unit. The strip pooling unit performs average pooling on the feature map of the input efficient local attention mechanism along the horizontal and vertical directions, respectively, to extract horizontal and vertical feature vectors that also possess positional encoding characteristics. The convolution unit performs one-dimensional convolution on the horizontal and vertical feature vectors to expand the local receptive field and enhance inter-directional interaction. The normalization and activation unit uses group normalization and the sigmoid activation function to generate horizontal and vertical attention weights on the feature map processed by the convolution unit. The weight fusion unit multiplies the two sets of weights point-by-point to obtain a comprehensive spatial attention weight, and performs element-wise multiplication with the original input feature map of the efficient local attention mechanism along the channel to complete feature enhancement.

5. The method for detecting wire lay defects based on direction-width prior enhancement and deep learning according to claim 1, characterized in that, The visualization output of defect detection results and closed-loop control include: building a graphical human-machine interface based on the Qt framework, synchronously overlaying and displaying the wire laying image, the defect mask obtained by YOLOv8 defect detection, and the minimum bounding rectangle of the defect, while recording defect parameters, and controlling the wire laying head to stop through online feedback.

6. A wire layup defect detection system based on direction-width prior enhancement and deep learning, characterized in that, include: The acquisition module is used to acquire images of the wire laying process. The preprocessing module is used to preprocess the wire laying image, including orientation prior, width prior, dark area difference, and bright area top cap; The model building module is used to replace the base sampling module in the original YOLOv8 network with an efficient upsampling module and add an efficient local attention mechanism to obtain an improved YOLOv8 network. The preprocessed wire-laying image is then input into the improved YOLOv8 network for defect detection. The high-efficiency upsampling module enlarges the size of the feature map input to the high-efficiency upsampling module through bilinear interpolation. An efficient local attention mechanism is used to enhance features on the feature map input to the efficient local attention mechanism; The visualization module is used to visualize the defect detection results and perform closed-loop control.

7. The wire layup defect detection system based on direction-width prior enhancement and deep learning according to claim 6, characterized in that, In the preprocessing module, the preprocessing of the fiber-laying image includes: The filament-laying image is subjected to a two-dimensional Fourier transform and centered. After masking the low frequency of the central circular domain, the angular energy distribution is obtained by integration. The peak angles of the first two layers are used to construct a directional filter. Based on the filtering directional filtering features, the directional prior image is obtained. A rectangular structural element is constructed to perform an opening operation on the orientation prior image to obtain a filtered image; the minor axis of the rectangular structural element is equal to the pixel width of the filament bundle in the orientation prior image, and the major axis is along the current filament laying direction. Perform dark area difference operation on the filtered image and simultaneously perform bright area top cap transformation to obtain the preprocessed wire-laying image.

8. The wire layup defect detection system based on direction-width prior enhancement and deep learning according to claim 6, characterized in that, In the model building module, the efficient upsampling module includes: an upsampling unit, a feature enhancement unit, a normalization and activation unit, and a channel adjustment unit. The upsampling unit uses bilinear interpolation to enlarge the feature map input to the efficient upsampling module by a factor of two, obtaining an expanded domain and aligning it with the spatial resolution. The feature enhancement unit uses a 3×3 depthwise separable convolution to extract local spatial features within the expanded domain. The normalization and activation unit sequentially performs batch normalization and ReLU activation on the feature map processed by the feature enhancement unit to normalize the distribution of the extracted local spatial features. The channel adjustment unit unifies the channel dimensions through a 1×1 pointwise convolution.

9. The wire layup defect detection system based on direction-width prior enhancement and deep learning according to claim 6, characterized in that, In the model building module, the efficient local attention mechanism includes: a strip pooling unit, a convolution unit, a normalization and activation unit, and a weight fusion unit. Specifically, the strip pooling unit performs average pooling on the feature map of the input efficient local attention mechanism along the horizontal and vertical directions, respectively, to extract horizontal and vertical feature vectors that possess both positional encoding characteristics. The convolution unit performs one-dimensional convolution on the horizontal and vertical feature vectors to expand the local receptive field and enhance inter-directional interaction. The normalization and activation unit uses group normalization and the sigmoid activation function to generate horizontal and vertical attention weights on the feature map processed by the convolution unit. The weight fusion unit multiplies the two sets of weights point-by-point to obtain a comprehensive spatial attention weight, and performs element-wise multiplication with the original input feature map of the efficient local attention mechanism along the channel to complete feature enhancement.

10. The wire layup defect detection system based on direction-width prior enhancement and deep learning according to claim 6, characterized in that, In the visualization module, the visualization output of defect detection results and closed-loop control include: building a graphical human-machine interface based on the Qt framework, synchronously overlaying and displaying the wire laying image, the defect mask obtained by YOLOv8 defect detection, and the minimum bounding rectangle of the defect, while recording defect parameters, and controlling the wire laying head to stop through online feedback.

Images