Composite material flaw automatic screening system and method based on machine vision

Abstract

This invention discloses an automatic defect screening system and method for composite materials based on machine vision. The system includes: a conveying mechanism for carrying and uniformly conveying the composite material to be inspected; and an image acquisition module, located at the inspection station of the conveying mechanism, including a multispectral imaging unit and a dynamically programmable light source unit, for acquiring surface and shallow internal images of the composite material under different light source excitation conditions and multiple spectral channels to obtain multimodal image data. This automatic defect screening system and method for composite materials based on machine vision possesses multimodal and multi-layer detection capabilities. By incorporating a multispectral imaging unit and a dynamically programmable light source unit, it can simultaneously acquire images of the surface morphology and shallow internal fiber distribution of the composite material under different spectral channels and illumination modes. The near-infrared band can effectively penetrate the surface resin, clearly revealing internal layering, pores, and fiber orientation, significantly improving the detection rate of hidden defects.

Description

Technical Field

[0001] This invention relates to the field of composite material inspection technology, specifically to an automatic screening system and method for composite material defects based on machine vision. Background Technology

[0002] Composite materials (such as carbon fiber reinforced composites and glass fiber reinforced composites) have been widely used in aerospace, automotive manufacturing, wind turbine blades, and pressure vessels due to their excellent properties such as high specific strength, high specific modulus, and corrosion resistance. However, during the preparation and processing of composite materials, internal and surface defects such as porosity, delamination, inclusions, fiber breakage, resin accumulation, or microcracks are easily generated. These defects, especially shallow internal defects and microcracks, can severely weaken the mechanical properties and service life of structural components, and even lead to catastrophic accidents.

[0003] Traditional manual visual inspection methods are inefficient, highly subjective, and unable to detect internal defects. While automated optical inspection (AOI) systems based on ordinary machine vision have improved inspection speed to some extent, they still face the following technical bottlenecks: 1. A single visible light source is difficult to penetrate the surface resin of opaque or semi-transparent composite materials, making it impossible to image the internal fiber distribution and delamination defects; 2. Composite material surfaces often have periodic woven textures or high light reflection, making conventional image processing algorithms prone to false detections; 3. Minor defects (such as microcracks and micro-inclusions) account for a very small proportion in the image, resulting in a severe imbalance between positive and negative samples and low recall rates for traditional detection models; 4. Existing detection systems are mostly static models, unable to utilize false / missed samples generated on-site for continuous optimization, and difficult to adapt to variations in material processing across different batches. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the existing defects and provide an automatic screening system and method for composite material defects based on machine vision. This system can simultaneously achieve high-precision screening of surface and shallow internal defects, effectively suppress texture and reflection interference, and has model iteration capability, thus effectively solving the problems in the background technology.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A machine vision-based automatic defect screening system for composite materials includes: A conveying mechanism is used to carry and transport the composite material to be tested at a constant speed. An image acquisition module, located at the detection station of the conveying mechanism, includes a multispectral imaging unit and a dynamic programmable light source unit, used to acquire surface and shallow internal images of the composite material under different light source excitation conditions and multiple spectral channels to obtain multimodal image data; The image preprocessing module is communicatively connected to the image acquisition module and is used to perform registration, denoising, and contrast enhancement processing on the multimodal image data to generate a standard detection image. The defect intelligent detection module has a built-in pre-trained deep neural network model, which is used to receive the standard detection image and output the pixel-level segmentation mask, defect category and confidence level of the defect through multi-scale feature extraction and spatial attention mechanism. The control and execution module is connected to the conveying mechanism and the defect intelligent detection module, respectively, and is used to generate control commands according to the defect category and confidence level, and drive the sorting device or marking device on the conveying mechanism to remove or mark the composite material with defects. The data management and model iteration module is used to store detection data and perform incremental learning and updates on the deep neural network model based on manually reviewed false detection samples and / or missed detection samples.

[0006] Preferably, the dynamic programmable light source unit includes a coaxial shadowless light source, a low-angle ring light source, and a polarized light source; the multispectral imaging unit includes a high-resolution linear array camera and a multispectral filter wheel, the multispectral filter wheel including visible light band filters and near-infrared band filters, used to penetrate the surface resin of the composite material to obtain images of internal fiber distribution and layered defects.

[0007] Preferably, the deep neural network model includes: The feature encoding subnetwork, employing a residual network structure with dilated convolutions, is used to extract multi-scale feature maps from the standard detection image. The feature fusion subnetwork includes a channel attention module and a spatial attention module, which are used to perform cross-layer splicing and weight adaptive allocation on the multi-scale feature maps to highlight the feature responses of micro-cracks and impurities. The feature decoding subnetwork employs a progressive upsampling structure combined with skip connections to output defect pixel-level segmentation results with the same resolution as the input image.

[0008] Preferably, the automatic defect screening system for composite materials based on machine vision further includes: an environmental compensation sensor for acquiring temperature and humidity data of the inspection station in real time; and an image preprocessing module with an embedded environmental compensation algorithm for adaptively calibrating the color shift and reflectivity of the multimodal image data based on the temperature and humidity data.

[0009] The automatic defect screening method for composite materials based on machine vision, applied to the aforementioned automatic defect screening system for composite materials based on machine vision, includes the following steps: S1: Control the dynamic programmable light source unit to switch the illumination mode according to a preset timing sequence, and simultaneously trigger the multispectral imaging unit to acquire multimodal image data of the composite material; S2: Perform multi-channel image registration and adaptive histogram equalization on the multimodal image data to eliminate surface texture interference of composite materials and generate a standard detection image; S3: Input the standard detection image into a pre-trained deep neural network model, extract multi-scale features and calculate the attention weight map to generate a defect semantic segmentation map; S4: Perform connected component analysis on the defect semantic segmentation graph, calculate the geometric parameters of the defect, and combine the classifier to output the defect category and severity level; S5: Based on the defect category and severity level, generate corresponding control signals to drive the actuator to complete the automatic screening and sorting of defective products.

[0010] Preferably, in step S3, the training process of the pre-trained deep neural network model includes: Construct a composite material image dataset containing normal samples and various defective samples, and perform pixel-level annotation; A joint loss function of Focal Loss and Dice Loss is introduced to address the problem of extreme imbalance between positive and negative samples caused by minor imperfections in composite material images. The network is iteratively optimized using a combination of stochastic gradient descent and cosine annealing learning rate strategy until the average accuracy of the model on the validation set reaches a preset threshold.

[0011] The aforementioned automatic defect screening method for composite materials based on machine vision, wherein step S2, eliminating surface texture interference of the composite material, specifically includes: A frequency domain filtering algorithm based on Fourier transform is used to identify and suppress the inherent periodic woven texture frequency components of composite materials. A Stokes vector calculation method based on polarized light imaging is used to separate and filter out the specular reflection light component of the composite material surface, while retaining the diffuse reflection light component.

[0012] Preferably, the geometric parameters in step S4 include: the area of ​​the defect, its perimeter, the aspect ratio of the minimum bounding rectangle, and the main direction angle; the severity level is determined by the following rule: when the defect area is greater than a preset area threshold, or when the angle between the main direction angle of the defect and the fiber layup direction of the composite material is within a preset danger range, it is determined to be a severe level.

[0013] Preferably, the conveying mechanism is equipped with a rotary encoder for real-time detection of conveying speed and displacement; the image acquisition module triggers the multispectral imaging unit to perform equidistant sampling based on the pulse signal of the rotary encoder, so as to ensure that the images acquired under different lighting conditions correspond accurately in space.

[0014] Preferably, after step S5, the method further includes: superimposing the defect region outline and label onto the original multimodal image data according to the defect semantic segmentation map, defect category and severity level, generating a visual detection result map, and outputting a detection report according to a preset format, wherein the detection report includes the number of defects, location coordinates and statistical distribution information.

[0015] Compared with the prior art, the beneficial effects of the present invention are: With multimodal and multi-layer detection capabilities, this system incorporates a multispectral imaging unit (visible light + near-infrared filter) and a dynamically programmable light source unit (coaxial shadowless, low-angle ring, polarized light source). It can simultaneously acquire images of the surface morphology and shallow internal fiber distribution of composite materials under different spectral channels and illumination modes. The near-infrared band effectively penetrates the surface resin, clearly revealing internal layering, pores, and fiber orientation, significantly improving the detection rate of hidden defects. High-precision pixel-level segmentation of defects: The deep neural network model uses a residual structure with dilated convolution to extract multi-scale features and combines channel attention and spatial attention mechanisms to perform cross-layer feature fusion. It can achieve sub-millimeter-level pixel-level segmentation of small target defects such as microcracks and impurities. Compared with traditional detection methods based on manual features, this invention has significantly improved recall and intersection-over-union (IoU). It has strong anti-texture and anti-reflection interference capabilities. The frequency domain filtering algorithm in the image preprocessing module can effectively suppress the inherent periodic woven texture frequency components of composite materials. At the same time, based on polarized light Stokes vector calculation, specular reflection light can be separated and filtered out, leaving only diffuse reflection light components. The combination of the two enables the system to work stably on composite materials with high gloss surfaces or complex textures. Environmental adaptive compensation, by setting temperature and humidity sensors and embedding environmental compensation algorithms, can calibrate in real time the image color shift and reflectivity fluctuation caused by temperature and humidity changes, ensuring detection consistency, especially suitable for production workshops without constant temperature and humidity conditions; Incremental learning and continuous model iteration: The data management and model iteration module can store all detection results and false / missed detection samples reviewed by manual verification, and uses incremental learning to periodically update the deep neural network. This enables the system to adapt to the process fluctuations of composite materials from different batches and suppliers, and the detection accuracy continues to improve after long-term operation. 6. Precise spatial synchronization and equidistant sampling: The rotary encoder on the conveying mechanism triggers the multispectral imaging unit to perform equidistant sampling according to the actual displacement, ensuring that the images acquired under different light source timing switching modes are completely spatially corresponding, avoiding registration errors caused by fluctuations in conveying speed. 7. Visualized and traceable test results: The method and steps overlay the defect outline and category label onto the original image and generate a test report containing the number of defects, location coordinates, and statistical distribution. This facilitates quick review and quality traceability by operators and meets the actual needs of industrial production lines for data retention and batch management. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating the architecture of the machine vision-based automatic defect screening system for composite materials according to the present invention.

[0017] Figure 2 This is a schematic diagram of the automatic defect screening method for composite materials based on machine vision according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Please see Figure 1-2 The present invention provides a technical solution: A machine vision-based automatic defect screening system for composite materials includes: A conveying mechanism is used to carry and transport the composite material to be tested at a constant speed. The image acquisition module, located at the inspection station of the conveying mechanism, includes a multispectral imaging unit and a dynamic programmable light source unit. It is used to acquire surface and shallow internal images of the composite material under different light source excitation conditions and multiple spectral channels to obtain multimodal image data. The image preprocessing module is connected to the image acquisition module and is used to perform registration, denoising, and contrast enhancement on multimodal image data to generate standard detection images. The defect intelligent detection module has a built-in pre-trained deep neural network model, which is used to receive standard detection images and output pixel-level segmentation masks, defect categories and confidence scores of defects through multi-scale feature extraction and spatial attention mechanism. The control and execution module is connected to the conveying mechanism and the defect intelligent detection module, respectively. It is used to generate control commands based on the defect type and confidence level, and drive the sorting device or marking device on the conveying mechanism to reject or mark the composite material with defects. The data management and model iteration module is used to store detection data and perform incremental learning and updates on the deep neural network model based on false detection samples and / or missed detection samples reviewed by humans.

[0020] The dynamic programmable light source unit includes a coaxial shadowless light source, a low-angle ring light source, and a polarized light source; the multispectral imaging unit includes a high-resolution linear array camera and a multispectral filter wheel, which includes visible light band filters and near-infrared band filters, used to penetrate the surface resin of the composite material to obtain images of internal fiber distribution and layered defects.

[0021] Deep neural network models include: The feature encoding subnetwork employs a residual network structure with dilated convolutions to extract multi-scale feature maps from standard detection images. The feature fusion subnetwork, which includes a channel attention module and a spatial attention module, is used to perform cross-layer splicing and adaptive weight allocation of multi-scale feature maps to highlight the feature responses of micro-cracks and impurities. The feature decoding subnetwork employs a progressive upsampling structure combined with skip connections to output defect pixel-level segmentation results with the same resolution as the input image.

[0022] The machine vision-based automatic defect screening system for composite materials also includes: an environmental compensation sensor for real-time acquisition of temperature and humidity data at the inspection station; and an image preprocessing module with an embedded environmental compensation algorithm for adaptive calibration of color shift and reflectivity of multimodal image data based on temperature and humidity data.

[0023] An automatic defect screening method for composite materials based on machine vision, applied to an automatic defect screening system for composite materials based on machine vision, includes the following steps: S1: Control the dynamic programmable light source unit to switch the illumination mode according to the preset timing, and simultaneously trigger the multispectral imaging unit to acquire multimodal image data of the composite material; S2: Perform multi-channel image registration and adaptive histogram equalization on multimodal image data to eliminate surface texture interference of composite materials and generate standard detection images; S3: Input the standard detection image into a pre-trained deep neural network model, extract multi-scale features and calculate the attention weight map to generate a defect semantic segmentation map; S4: Perform connected component analysis on the defect semantic segmentation graph, calculate the geometric parameters of the defect, and combine with the classifier to output the defect category and severity level; S5: Based on the defect category and severity level, generate corresponding control signals to drive the actuator to complete the automatic screening and sorting of defective products.

[0024] In step S3, the training process of the pre-trained deep neural network model includes: Construct a composite material image dataset containing normal samples and various defective samples, and perform pixel-level annotation; A joint loss function of Focal Loss and Dice Loss is introduced to address the problem of extreme imbalance between positive and negative samples caused by minor imperfections in composite material images. The network is iteratively optimized using a combination of stochastic gradient descent and cosine annealing learning rate strategy until the average accuracy of the model on the validation set reaches a preset threshold.

[0025] Step S2, which eliminates surface texture interference in the composite material, specifically includes: A frequency domain filtering algorithm based on Fourier transform is used to identify and suppress the inherent periodic woven texture frequency components of composite materials. A Stokes vector calculation method based on polarized light imaging is used to separate and filter out the specular reflection light component of the composite material surface, while retaining the diffuse reflection light component.

[0026] The geometric parameters in step S4 include: the area of ​​the defect, its perimeter, the aspect ratio of the minimum bounding rectangle, and the main direction angle; the severity level is determined as follows: when the defect area is greater than the preset area threshold, or when the angle between the main direction angle of the defect and the fiber layup direction of the composite material is within the preset danger range, it is determined to be severe.

[0027] The conveying mechanism is equipped with a rotary encoder for real-time detection of conveying speed and displacement; the image acquisition module triggers the multispectral imaging unit to perform equidistant sampling based on the pulse signal of the rotary encoder to ensure that the images acquired under different lighting conditions correspond accurately in space.

[0028] Step S5 and beyond also includes: based on the defect semantic segmentation map, defect category and severity level, overlaying the defect area outline and label onto the original multimodal image data to generate a visual detection result map, and outputting a detection report in a preset format, the detection report includes the number of defects, location coordinates and statistical distribution information.

[0029] System overall structure: An automatic defect screening system for composite materials based on machine vision includes a conveying mechanism, an image acquisition module, an image preprocessing module, an intelligent defect detection module, a control and execution module, and a data management and model iteration module.

[0030] Conveying Mechanism: A flat belt conveyor with variable frequency speed control is used. The belt surface is made of matte black material to reduce glare. A rotary encoder (such as an incremental photoelectric encoder) is installed on the conveying mechanism, and its rollers are coaxially connected to the drive roller of the conveyor belt. The encoder outputs A / B phase pulse signals to detect the conveying speed and displacement in real time.

[0031] Image acquisition module: Located at the inspection station in the middle of the conveying mechanism. The dynamically programmable light source unit is illuminated sequentially by the controller according to a preset time sequence: a coaxial shadowless light source (for uniformly illuminating surface textures), a low-angle ring light source (for highlighting edges and micro-cracks), and a polarized light source (for suppressing specular reflection). The multispectral imaging unit is a high-resolution linear array camera (pixel size ≤ 5μm), with a multispectral filter wheel installed in front of its lens. Visible light filters (400-700nm) and near-infrared filters (780-1100nm) are alternately arranged on the filter wheel. The camera's external trigger signal comes from the pulse signal of the rotary encoder, achieving equidistant sampling (e.g., acquiring one line of images every 0.05mm).

[0032] Image preprocessing module: Employing an FPGA+ARM heterogeneous computing platform, it receives multimodal image data from the camera in real time. First, multi-channel image registration is performed: Due to sub-pixel shifts in images under different lighting conditions, a rigid registration algorithm based on mutual information is used to align the images across channels. Then, frequency domain filtering is performed: A two-dimensional Fast Fourier Transform (FFT) is applied to the image to identify bright spots (frequency peaks) corresponding to the periodic texture of the composite material on the spectrogram. These frequency components are suppressed using a band-stop filter, and then an inverse transform is performed to obtain a detextured image. Simultaneously, based on two orthogonally polarized images acquired under polarized light sources, Stokes vectors (S0, S1) are calculated to obtain the diffuse reflection light component image. Finally, adaptive histogram equalization (CLAHE) is performed to enhance contrast and generate a standard detection image.

[0033] Environmental compensation sensor: A temperature and humidity integrated sensor is installed next to the camera to collect the temperature (°C) and relative humidity (%RH) of the detection station in real time. The image preprocessing module has a built-in environmental compensation algorithm, such as pre-calibrating the RGB response of a standard gray board under different temperatures and humidity, establishing a multinomial regression model, and correcting image color shift in real time; at the same time, it corrects the specular reflection light intensity threshold according to humidity.

[0034] The defect detection module is deployed on a GPU server (such as NVIDIA Jetson Orin) and internally loads a pre-trained deep neural network model. The specific structure of this model is as follows: Feature encoding subnetwork: ResNet-50 is used as the backbone, and some standard convolutions in layers 3, 4 and 5 are replaced with dilated convolutions (dilation=2,4) to increase the receptive field without reducing the feature map resolution.

[0035] Feature fusion subnetwork: After the encoder, channel attention module (SENet) and spatial attention module (similar to CBAM) are connected to perform weighted fusion of multi-scale feature maps to highlight the slender response of microcracks.

[0036] Feature decoding subnetwork: It adopts progressive upsampling (first upsampled by 2x, then by 2x, and finally upsampled to the original image size). After each upsampling stage, it is concatenated with the corresponding layer of the encoder through skip connections. The output layer is a 1×1 convolution with a sigmoid activation function, which produces a pixel-level segmentation mask for defects (0 for background, 1 for defects). At the same time, it outputs the defect category (such as "pore", "layering", "crack", "inclusion") and confidence score through another branch.

[0037] Control and Execution Module: A PLC (Programmable Logic Controller) receives the detection results from the intelligent defect detection module. When a defect is detected and the confidence level is >0.7, control instructions are generated based on the defect category and severity level: For severe defects, a pneumatic pusher is driven to push the defective composite material into the waste bin (rejection); for minor defects, an inkjet printer is driven to mark the position on the product edge (marking). The PLC is also linked with the frequency converter of the conveyor motor to achieve defect location tracking.

[0038] Data management and model iteration module: Employs an industrial computer and a database (SQLite). All detected images, segmentation masks, defect information, and corresponding temperature, humidity, and light source parameters are stored. Operators manually verify the detection results through a verification terminal, marking false positives and false negatives. After accumulating 500 verification samples, the system automatically triggers incremental learning: using the parameters of the old model as initial weights, it performs several rounds of fine-tuning only on new samples (and some memory playback samples) to update the deep neural network model.

[0039] Specific implementation steps of the detection method: The following is a specific embodiment of a machine vision-based automatic defect screening method for composite materials, based on the above system.

[0040] Step S1: Multimodal Image Acquisition The conveyor mechanism is activated, and the composite material to be tested (e.g., carbon fiber prepreg laminate) is laid flat on the conveyor belt. The system controls the dynamically programmable light source unit to cycle sequentially: during time period T0, only the coaxial shadowless light source is turned on, simultaneously triggering the linear scan camera under the near-infrared filter to acquire one line of images; during time period T1, only the low-angle ring light source is turned on, triggering the acquisition of one line under the visible light filter; during time period T2, only the polarized light source (0° polarizer) is turned on, acquiring one line; during time period T3, the polarized light source (90° polarizer) is turned on, acquiring one line. The four lines of acquisition correspond to the same physical position, and the rotary encoder pulses ensure spatial synchronization. This continuous acquisition process ultimately yields four complete multimodal images.

[0041] Step S2: Image Preprocessing First, multi-channel registration is performed: due to mechanical errors and light source switching delays, there are slight offsets in the four images. The offset is calculated using the mutual information method and resampled. Then, frequency domain filtering is performed: an FFT is performed on the visible light image to identify the frequency peaks corresponding to the carbon fiber weaving cycle (e.g., 6 cycles per millimeter), and a Gaussian band-stop filter (center frequency ± bandwidth) is designed to suppress them. Next, polarization separation is performed: based on the images T2 and T3, the Stokes vectors S0 = I0° + I90° and S1 = I0° - I90° are calculated, and the diffuse reflection component is approximately S0 - |S1|. Then, the CLAHE algorithm (block size 64×64, cropping limit 2.0) is used to enhance contrast. Finally, based on the current temperature and humidity sensor values, a pre-stored color correction matrix (CCM) is called to perform a linear transformation on the RGB three channels to obtain the standard detection image.

[0042] Step S3: Deep Neural Network Inference The standard detection image is input into a pre-trained deep neural network model. The model first outputs multi-scale feature maps (1 / 4, 1 / 8, and 1 / 16 the size of the original image) through an encoding sub-network. Then, a feature fusion sub-network calculates channel attention weights and spatial attention weights, which are then weighted and fed into the decoding sub-network. The decoder outputs a defect semantic segmentation map at the same resolution as the input (the value of each pixel represents its probability of being a defect). Simultaneously, the segmentation map undergoes global average pooling and a fully connected layer to output the defect category (e.g., "crack").

[0043] Step S4: Connectivity Analysis and Severity Determination Thresholding (threshold 0.5) is applied to the defect semantic segmentation map to obtain a binary mask. 8-connected component analysis is used to extract each connected component. The geometric parameters of each connected component are calculated. Area (number of pixels, multiplied by a calibration factor to obtain the actual area in mm²); Perimeter (length of boundary pixels); The aspect ratio of the smallest bounding rectangle; Principal direction angle (calculated using the minimum bounding rectangle or PCA).

[0044] The classifier (e.g., a lightweight random forest) outputs the final defect category based on geometric parameters and category information. The severity level is determined as follows: if the defect area is greater than a preset area threshold (e.g., 2 mm²), or the angle between the defect's main direction angle and the composite fiber layup direction is within a preset danger range (e.g., 45°±15°), it is classified as "severe"; otherwise, it is classified as "minor".

[0045] Step S5: Control and Diversion The control and execution module generates control signals based on the defect category and severity level. For example, for "severe" defects categorized as "delamination" or "cracks," after a certain delay (position tracking via encoder pulse counting), the pneumatic pusher is driven to push the sheet into the waste bin; for "minor" defects, the inkjet printer is driven to print a dot at the corresponding position on the product edge.

[0046] Additional steps: After all inspections are completed, the system generates a visual inspection result image: the outline of the defect semantic segmentation map is overlaid on the original visible light image with red lines, and labeled with text such as "Crack - Severe". Simultaneously, a JSON-formatted inspection report is output, including: product batch number, inspection time, total number of defects, absolute coordinates of each defect (based on the transport start point and encoder cumulative pulses), and defect type statistical distribution. This report can be uploaded to the MES system or printed locally.

[0047] Specific implementation methods of model training For the pre-training of deep neural network models, the present invention provides the following preferred solutions: Dataset Construction: 10,000 images of composite materials (including normal and various defective images) were collected and pixel-level annotated by professional quality inspectors using annotation tools (such as LabelMe). Data Augmentation: Random rotation, scaling, adding Gaussian noise, and simulating changes in light source intensity.

[0048] Loss function: A joint loss function L = α * Focal Loss + β * Dice Loss is used, where α=1 and β=1. The Focal Loss parameter γ=2 to reduce the weight of easily classified negative samples and focus on minor flaws. The Dice Loss directly optimizes the segmentation intersection-union ratio.

[0049] Optimization strategy: SGD optimizer was used with an initial learning rate of 0.01, momentum of 0.9, and weight decay of 0.0005. Cosine annealing learning rate scheduling was employed (T_max = 10 epochs). Training was performed on 8 NVIDIA Tesla T4 GPUs with a batch size of 16, for a total of 200 epochs. Training stopped when the mAP (mean precision, IoU threshold 0.5) on the validation set reached 0.92 or higher.

[0050] Incremental learning: During each incremental update, 10% of the old samples are randomly selected from the historical samples (to prevent catastrophic forgetting) and combined with newly added review samples (usually 20-50 images per category) to form a mini training set. Initialized with the current model weights, iterate for 10-20 rounds with a small learning rate (0.001). Example

[0051] As a preferred embodiment, two inspection stations can be set up in parallel on the conveying mechanism: the first station uses the aforementioned system to perform top inspection, and the second station uses a flipping mechanism to inspect the back of the composite material, achieving full coverage on both sides.

[0052] As another preferred implementation, the output segmentation mask of the deep neural network model can also be used for the three-dimensional reconstruction of defects: the depth information of defects is estimated by combining the difference in the penetration depth of near-infrared light in multispectral imaging with optical flow method.

[0053] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An automatic defect screening system for composite materials based on machine vision, characterized in that, include: A conveying mechanism is used to carry and transport the composite material to be tested at a constant speed. An image acquisition module, located at the detection station of the conveying mechanism, includes a multispectral imaging unit and a dynamic programmable light source unit, used to acquire surface and shallow internal images of the composite material under different light source excitation conditions and multiple spectral channels to obtain multimodal image data; The image preprocessing module is communicatively connected to the image acquisition module and is used to perform registration, denoising, and contrast enhancement processing on the multimodal image data to generate a standard detection image. The defect intelligent detection module has a built-in pre-trained deep neural network model, which is used to receive the standard detection image and output the pixel-level segmentation mask, defect category and confidence level of the defect through multi-scale feature extraction and spatial attention mechanism. The control and execution module is connected to the conveying mechanism and the defect intelligent detection module, respectively, and is used to generate control commands according to the defect category and confidence level, and drive the sorting device or marking device on the conveying mechanism to remove or mark the composite material with defects. The data management and model iteration module is used to store detection data and perform incremental learning and updates on the deep neural network model based on manually reviewed false detection samples and / or missed detection samples.

2. The automatic defect screening system for composite materials based on machine vision according to claim 1, characterized in that, The dynamic programmable light source unit includes a coaxial shadowless light source, a low-angle ring light source, and a polarized light source; the multispectral imaging unit includes a high-resolution linear array camera and a multispectral filter wheel, the multispectral filter wheel including visible light band filters and near-infrared band filters, used to penetrate the surface resin of the composite material to obtain images of internal fiber distribution and layered defects.

3. The automatic defect screening system for composite materials based on machine vision according to claim 1, characterized in that, The deep neural network model includes: The feature encoding subnetwork, employing a residual network structure with dilated convolutions, is used to extract multi-scale feature maps from the standard detection image. The feature fusion subnetwork includes a channel attention module and a spatial attention module, which are used to perform cross-layer splicing and weight adaptive allocation on the multi-scale feature maps to highlight the feature responses of micro-cracks and impurities. The feature decoding subnetwork employs a progressive upsampling structure combined with skip connections to output defect pixel-level segmentation results with the same resolution as the input image.

4. The automatic defect screening system for composite materials based on machine vision according to claim 1, characterized in that, Also includes: An environmental compensation sensor is used to acquire real-time temperature and humidity data at the detection station. The image preprocessing module has an embedded environmental compensation algorithm, which is used to adaptively calibrate the color shift and reflectivity of the multimodal image data based on the temperature and humidity data.

5. A machine vision-based automatic defect screening method for composite materials, applied to any one of the machine vision-based automatic defect screening systems for composite materials according to claims 1-4, characterized in that, Includes the following steps: S1: Control the dynamic programmable light source unit to switch the illumination mode according to a preset timing sequence, and simultaneously trigger the multispectral imaging unit to acquire multimodal image data of the composite material; S2: Perform multi-channel image registration and adaptive histogram equalization on the multimodal image data to eliminate surface texture interference of composite materials and generate a standard detection image; S3: Input the standard detection image into a pre-trained deep neural network model, extract multi-scale features and calculate the attention weight map to generate a defect semantic segmentation map; S4: Perform connected component analysis on the defect semantic segmentation graph, calculate the geometric parameters of the defect, and combine the classifier to output the defect category and severity level; S5: Based on the defect category and severity level, generate corresponding control signals to drive the actuator to complete the automatic screening and sorting of defective products.

6. The automatic defect screening method for composite materials based on machine vision according to claim 5, characterized in that, In step S3, the training process of the pre-trained deep neural network model includes: Construct a composite material image dataset containing normal samples and various defective samples, and perform pixel-level annotation; A joint loss function of Focal Loss and Dice Loss is introduced to address the problem of extreme imbalance between positive and negative samples caused by minor imperfections in composite material images. The network is iteratively optimized using a combination of stochastic gradient descent and cosine annealing learning rate strategy until the average accuracy of the model on the validation set reaches a preset threshold.

7. The automatic defect screening method for composite materials based on machine vision according to claim 5, characterized in that, The step S2, eliminating surface texture interference of the composite material, specifically includes: A frequency domain filtering algorithm based on Fourier transform is used to identify and suppress the inherent periodic woven texture frequency components of composite materials. A Stokes vector calculation method based on polarized light imaging is used to separate and filter out the specular reflection light component of the composite material surface, while retaining the diffuse reflection light component.

8. The automatic defect screening method for composite materials based on machine vision according to claim 5, characterized in that, The geometric parameters in step S4 include: the area of ​​the defect, its perimeter, the aspect ratio of the minimum bounding rectangle, and the main direction angle; the severity level is determined by the following rule: when the defect area is greater than a preset area threshold, or when the angle between the main direction angle of the defect and the fiber layup direction of the composite material is within a preset danger range, it is determined to be a severe level.

9. The automatic defect screening method for composite materials based on machine vision according to claim 1, characterized in that, The conveying mechanism is equipped with a rotary encoder for real-time detection of conveying speed and displacement; the image acquisition module triggers the multispectral imaging unit to perform equidistant sampling based on the pulse signal of the rotary encoder, so as to ensure that the images acquired under different lighting conditions correspond accurately in space.

10. The automatic defect screening method for composite materials based on machine vision according to claim 5, characterized in that, After step S5, the method further includes: based on the defect semantic segmentation map, defect category and severity level, superimposing the defect area outline and label onto the original multimodal image data to generate a visual detection result map, and outputting a detection report in a preset format, the detection report including defect quantity, location coordinates and statistical distribution information.

Images