Composite material plate pit and dirt detection method based on machine vision

Abstract

The invention relates to the technical field of plate defect detection, in particular to a composite material plate pit and smudginess detection method based on machine vision, which comprises the following steps: S1, starting a visual system of a sensing layer, and continuously photographing a composite material plate by the visual system; s2, a visual program of the data and algorithm processing layer calls an image collected by the sensing layer, and after preprocessing, a target detection model integrated with an ECA channel attention mechanism is called to detect defects on the image; s3, the transmission layer outputs the category and position information of the defect to the application layer; s4, an in-layer labeling machine is applied to execute labeling operation; and S5, if the composite material plate exists in the visual field, the steps S2-S4 are circulated. According to the method, the deep learning model is trained by using the YOLOX algorithm, the model processing speed is high, the precision is relatively high, and an ECA channel attention mechanism is integrated in the YOLOX algorithm, so that the detection precision of the deep learning model can be further improved.

Description

Technical Field

[0001] The invention relates to the technical field of plate defect detection, and in particular to a method for detecting pits and dirt on a composite material plate based on machine vision. Background Art

[0002] Composite materials are new materials with higher performance such as high strength, light weight, corrosion resistance, wear resistance, and high temperature resistance, which are formed by combining two or more materials with different properties. They are widely used in aerospace, shipbuilding, automobiles, etc. Unfortunately, composite materials will inevitably have defects such as pits, fiber corrugations, fiber breaks, and dirt during the manufacturing process. Effectively detecting defects is of great significance to improving the quality of product delivery. Therefore, it is necessary to study a method for detecting pits and dirt in composite panels based on machine vision.

[0003] According to the current patent disclosures and literature, the publication number: CN118967697A discloses a method for detecting aluminum alloy plates for automobiles. An improved AlexNet network model including 6 convolutional stacking modules and 3 fully connected layers is constructed. Multiple images of aluminum alloy plates with known defects and multiple qualified images of aluminum alloy plates for automobiles are collected, and corresponding labels are assigned to the collected images to obtain a sample set. After training the improved AlexNet network model, an automobile aluminum alloy plate detection model is obtained. The aluminum alloy plate images are detected using the model and the detection results are output. Although this method can also detect plate defects, the AlexNet network model has a slow detection speed and is inefficient for different specifications of aluminum alloys. The metal plate needs to adjust the camera focal length frequently, the deployment efficiency is low, and it is difficult to adapt to the assembly line real-time detection conditions; Publication No.: CN117036259A discloses a metal plate surface defect detection method based on deep learning. First, a surface image covering the entire metal plate is obtained and grayscale, noise reduction and other processing are performed, and then a training set is made and the training set is used to train the YOLOV5 target detection model. Finally, the model is used to perform defect detection on the metal plate defects. This method can also complete the defect detection of the plate, but in the face of the defect morphology of the plate with changeable shapes, the data set has not been enhanced, and the target detection algorithm has not been optimized. This will lead to the limited accuracy of the trained target detection model, which is difficult to detect when faced with subtle defects, resulting in missed detection. Publication number: CN111307825B A method for detecting pit defects on the surface of a wooden board includes a horizontally transmitted wooden board, a linear light source located on one side of a shading baffle and emitting light toward the bottom of the shading baffle, a camera located on the other side of the shading baffle and with a lens toward the bottom of the shading baffle, and a computer connected to the camera. This invention is not suitable for detecting boards of various specifications, the degree of automation of the detection process is not high, and there is a problem of insufficient detection accuracy.

[0004] In the industrial world, the accuracy and efficiency of industrial product defect detection directly determine its delivery capacity and delivery quality. Keeping a good checkpoint on industrial product defect detection is of great significance to improving product competitiveness. However, in summary, although the existing plate defect detection methods and systems can realize the detection of plate defects to a certain extent, there are still many shortcomings.

[0005] Therefore, we studied a method for detecting pits and dirt on composite panels based on machine vision, which provides an effective technical approach to realize panel defect detection and solves the problems of low detection efficiency and insufficient detection accuracy of existing panel defect detection systems and methods. Summary of the invention

[0006] In view of this, the present invention aims to propose a method for detecting pits and dirt in composite material plates based on machine vision, so as to solve the problems of low detection efficiency and insufficient detection accuracy of existing plate defect detection methods.

[0007] In order to solve the above problems, the present application firstly builds a camera assembly equipped with a liquid lens and a programmable stripe light, so that the focal length and lighting parameters can be automatically adjusted for composite material boards of different specifications, so as to obtain a clear original image with more obvious defects; secondly, the original image is filtered and contrast enhanced by using Gaussian function and Contrast function respectively, so that the image effect is better and the defects are further highlighted; then, in the early stage of deep learning model training, defect samples are derived by generating adversarial networks suitable for composite material board defects, the deep learning training set is enriched, and the detection accuracy of the deep learning model is improved; then, the YOLOX algorithm is used to train the deep learning model, and the model has fast processing speed and high accuracy. In addition, the ECA channel attention mechanism is integrated into the YOLOX algorithm, which can further improve the detection accuracy of the deep learning model. Finally, the transmission layer transmits the defect type and location information to the application layer, and the application layer labeler marks the defects on the surface of the composite material board to facilitate subsequent processing. The whole process is highly automated, from image acquisition, defect detection to labeling, reducing the need for manual intervention, not only can the existence of defects be detected, but also their location and category can be accurately determined, providing important information for subsequent processing.

[0008] The technical solution of the present invention is achieved in this way:

[0009] The present invention discloses a method for detecting pits and dirt on a composite material plate based on machine vision, comprising the following steps:

[0010] S1: Start the visual system of the perception layer, and the visual system takes pictures of the composite material board continuously;

[0011] S2: The visual program in the data and algorithm processing layer calls the image collected by the perception layer, first performs smoothing Gaussian filtering and contrast enhancement preprocessing, and then calls the YOLOX target detection model that incorporates the ECA channel attention mechanism to detect pits and dirt defects on the image;

[0012] S3: If the visual program detects that there are no defects on the composite material plate, the guide roller in the transmission layer continues to transmit the composite material plate, the perception layer continues to take continuous photos of the composite material plate, and the visual program continues to detect defects. If defects are detected on the composite material plate, the guide roller transmission is suspended, and the transmission layer outputs the defect category and location information to the application layer;

[0013] S4: The labeling machine in the application layer performs labeling based on the information output by the transmission layer, and classifies and labels the defects at the defect locations of the composite material plate;

[0014] S5: The guide rollers in the transmission layer continue to transport the composite material plates, and the sensing layer continues to collect images. If the sensing layer recognizes that there are no composite material plates in the field of view, it stops collecting images and ends the detection. If the sensing layer recognizes that there are still composite material plates passing through the field of view, it continues to collect images and loops through steps S2-S4 until all composite material plates are detected.

[0015] Further, in step S1, the perception layer includes a visual system and a beamforming optical fiber sensor, the visual system includes an industrial camera, a lens, a light source, an industrial computer, a light source controller and a transmission cable, and the specific steps of setting the perception layer to continuously take pictures of the composite material board are as follows:

[0016] S11. Determine the triggering position for taking photos: install a fiber optic sensor on the frame, adjust the position of the fiber optic sensor, and ensure that the signal is accurate when the fiber optic sensor detects that a composite material plate flows over the guide roller;

[0017] S12, deploying the camera assembly: the light emitting surface of the light source is set at 45 degrees to the composite material plate on the guide roller, the industrial camera and the lens are connected by threads, the axis of the lens is perpendicular to the composite material plate, and the industrial camera, the lens and the light source are all fixed on the horizontal bracket;

[0018] S13, component connection and control: the industrial camera is electrically connected to the POE network port of the industrial computer, the light source is electrically connected to the light source controller, the industrial computer, the light source controller, the optical fiber sensor and the PLC are electrically connected and the signals are exchanged;

[0019] S14, camera component parameter setting: the light source is a program-controlled stripe light, and the light source parameters are set by the light source controller, including brightness, stripe width, stripe direction and stripe moving speed; the camera parameters are set by the industrial camera imaging software NI MAX, including exposure time, contrast and Gama value;

[0020] S15, camera component imaging: When the through-beam fiber sensor detects that a composite material plate passes over the guide roller at the triggering position for photographing, the material information is sent to the PLC in the form of a Boolean quantity through the IO module. The PLC sends a photographing signal to the industrial computer through Modbus TCP. The imaging software NI MAX of the visual program on the industrial computer calls the industrial camera driver to capture images at a frequency of 15 frames per second. The latter image captured overlaps with the previous image, and there is no blind spot in the field of view. The visual program calls the image captured by the imaging software NI MAX in real time and stores the original image in a fixed folder of the industrial computer according to the date and product material number.

[0021] Specifically, the PLC drives the guide roller to rotate, thereby driving the composite material plate on the guide roller to move into the field of view of the industrial camera. The PLC drives the industrial camera to take pictures continuously at a frequency of 15 frames per second through the NI MAX imaging software of the LabView software platform on the industrial computer. The last picture collected has an overlapping part with the previous picture, and there is no blind spot in the field of view.

[0022] Furthermore, in step S2, the LabView language is used to write a visual program, and the data and algorithm processing layer processes the composite material board image collected by the perception layer. The specific steps are as follows:

[0023] S21. Image preprocessing: After the LabView visual program obtains the original image of the composite material plate, the Filters filter-Gaussian function in the tool software Vision Assistant of the LabView visual program is used to filter the original image, and the Contrast function is used to perform contrast enhancement on the filtered image;

[0024] S22. Load the target detection model: Load the YOLOX target detection model that incorporates the ECA channel attention mechanism in the LabView vision program;

[0025] S23. Implement defect detection: The YOLOX target detection model embedded in the LabView visual program implements defect detection on the pre-processed composite material plate image, marks the defect type and confidence on the image, and outputs the pixel position coordinates of the defect;

[0026] S24, result statistics and storage: LabView visual program accumulates and counts the number of occurrences of each defect type and displays it on the program interface. The pictures marked with the test results are stored in a fixed folder of the industrial computer according to the date, product number, and defect type.

[0027] Among them: When using the YOLOX algorithm to train the target detection model, a dynamic learning rate is used, and the dynamic adjustment formula is:

[0028]

[0029] Where: is the current learning rate, is the learning rate after iteration.

[0030] Furthermore, in step S3, the steps of the PLC in the transmission layer receiving the defect information sent by the data and algorithm processing layer and performing corresponding actions are as follows:

[0031] S31, transmission of detection results: LabView vision program sends the defect types detected by YOLOX target detection model to PLC via Modbus TCP with different feedback signals;

[0032] S32, coordinate conversion and transmission: LabView vision program converts the defect pixel position coordinates output by the YOLOX target detection model into world position coordinates after being processed by the calibration file, and sends the world position coordinates to the PLC in the form of data volume;

[0033] S33. Classification and position labeling: Based on the feedback signal sent by the LabView visual program and the world position coordinates of the defect, the PLC controls the multiple execution terminals of the labeling machine to classify and label the defects. The labels are divided into two colors, yellow for repairable defects and red for unrepairable defects, and the text of the defect type is printed on each label.

[0034] Furthermore, in step S1, the thickness of the composite material board has various specifications, and the lens is a liquid lens. The liquid lens stores the imaging focal length by point and is bundled with the product material number to ensure clear imaging when the distance from the lens to the surface to be inspected of the composite material board changes.

[0035] Furthermore, in step S1, a programmable stripe light source is selected, and the light source parameters are adjusted to take photos of the composite material plate multiple times under different lighting effects, and then the images are synthesized.

[0036] Furthermore, in step S2, the YOLOX target detection model uses a generative adversarial network to perform data expansion processing on the original defect samples in the early stage of deep learning training. The generative adversarial network maximizes the probability of distinguishing real data and generated data by training the discriminator D, and trains the generator G to minimize it. The optimization function is:

[0037]

[0038] The gradient of the discriminator D is:

[0039]

[0040] The gradient of the generator G is:

[0041]

[0042] The loss functions of the generative adversarial network discriminative model and generative model are:

[0043]

[0044]

[0045] In the formula, represents the objective function; , Represent the mathematical expectations of real data x and noise data z respectively; Represents the output of the generator; , Respectively represent the output of the discriminator D corresponding to the real data and the generated data; and They represent the gradient of the discriminator D and the generator G respectively; m represents the number of samples; i and j represent the sample numbers; x and z represent the real data and noise respectively.

[0046] Furthermore, in step S2, the pth standard model file trained by YOLOX is first converted into an onnx file, and then LabView is used to call the onnx target detection model of YOLOX to perform defect detection on the composite material plate.

[0047] Furthermore, in step S2, the function of the ECA channel attention mechanism is:

[0048]

[0049]

[0050] In the formula, are the input features, is the feature after stacking on the new dimension, is the final generated attention feature, and is the tensor average pooling function, is the tensor maximum pooling function, The convolution kernel size is The convolution operation is used to change the channel of the feature. yes Activation function, is the feature resizing function,

[0051] The size of the convolution kernel The channel dimension C can be adapted, the formula is:

[0052] .

[0053] Furthermore, in step S3, the area within the camera field of view is calibrated using Zhang’s calibration method, and the pixel coordinates of the defect position are converted into world coordinates that guide the labeling machine to label.

[0054] Compared with the prior art, the present invention has the following advantages in detecting pits and dirt on composite material plates based on machine vision:

[0055] 1. The present invention uses the YOLOX algorithm to train the deep learning model, and the model has fast processing speed and high accuracy. The ECA channel attention mechanism is integrated into the YOLOX algorithm, and a new weight function is set to further improve the detection accuracy of the deep learning model. At the same time, the system can detect and automatically handle defects in real time, reducing the downtime of the production line and improving production efficiency. The system has a high degree of automation, reduces manual intervention, and significantly improves detection efficiency.

[0056] 2. The present invention builds a camera assembly equipped with a liquid lens and a programmable stripe light, so that when facing composite material plates of different specifications, the focal length and lighting parameters can be automatically adjusted to obtain clear original images with more obvious defects.

[0057] 3. The present invention uses Gaussian function and Contrast function to filter and enhance the contrast of the original image, so that the image effect is better and the defects are further highlighted.

[0058] 4. In the early stage of deep learning model training, the present invention generates adversarial network-derived defect samples, sets a new GAN model loss function, enriches the deep learning training set, and improves the detection accuracy of the deep learning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0060] Figure 1 It is an execution flow chart of the present invention;

[0061] Figure 2 is a flow chart of the deployment and execution of the perception layer in the present invention;

[0062] Figure 3 It is a flow chart of data and algorithm processing in the present invention;

[0063] Figure 4 is a flow chart of result reception and application in the present invention;

[0064] Figure 5 It is a structural schematic diagram of the device in the present invention;

[0065] Figure 6 It is a network structure diagram of the YOLOX algorithm of the present invention that incorporates the ECA channel attention mechanism;

[0066] Figure 7 This is the effect diagram of the program-controlled stripe light of the present invention;

[0067] Figure 8 It is a rendering of the defect derived from the generative adversarial network of the present invention;

[0068] Fig. 9 It is the test result diagram of the present invention;

[0069] Fig.10 It is a software interface diagram of the present invention.

[0070] 1. Frame; 2. Guide roller; 3. Composite material plate; 4. Horizontal bracket; 5. PLC; 6. Optical fiber sensor; 7. Industrial camera; 8. Lens; 9. Light source; 10. Light source controller; 11. Labeling machine; 12. Industrial computer. DETAILED DESCRIPTION

[0071] In order to make the technical means, objectives and effects of the present invention easy to understand, the embodiments of the present invention are described in detail below with reference to specific drawings.

[0072] It should be noted that all the terms used in the present invention to indicate directionality and positionality, such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "inside", "outside", "top", "lower", "lateral", "longitudinal", "center", etc., are only used to explain the relative positional relationship and connection status between the components in a certain state, and are only for the convenience of describing the present invention, rather than requiring the present invention to be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes, and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features.

[0073] In the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0074] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0075] YOLOX is an important improved version of the YOLO (You Only Look Once) target detection model series, proposed by the Megvii Research team. YOLOX maintains the real-time performance of the YOLO series, and significantly improves the detection accuracy through multiple innovations such as Anchor-Free design, decoupling head, and dynamic label allocation, becoming one of the benchmark models in the current target detection field.

[0076] The Generative Adversarial Network (GAN) was proposed by Ian Goodfellow and others. It is a deep learning model that generates realistic data through adversarial training. It learns to generate data similar to the distribution of real data through the game between two neural networks, the generator G and the discriminator D. Specifically: the generator G tries to generate fake data that is as realistic as possible to deceive the discriminator D. The discriminator D tries to distinguish between real data and fake data generated by the generator G. In this process, the generator G and the discriminator D compete with each other and make progress together, and finally reach a Nash equilibrium. In this state, the data generated by the generator G cannot be effectively distinguished from the real data by the discriminator D.

[0077] ECA (Efficient Channel Attention) is a method for enhancing the efficiency of information interaction between channels in convolutional neural networks. It aims to capture the dependencies between channels by introducing a lightweight attention mechanism, thereby strengthening important feature representations while minimizing the amount of computation and the number of parameters.

[0078] LabVIEW vision programs are applications written using the LabVIEW software platform developed by National Instruments (NI) and are specifically used for machine vision tasks. LabVIEW is a graphical programming language that is widely used in test and measurement, data acquisition, instrument control, and machine vision.

[0079] The material number table refers to an Excel parameter file bundled with the material number. Each material number has a unique material number. The parameters bundled with the material number include the focal length of the liquid lens, the brightness of the light source, the parameters of the deep learning model, etc. Therefore, each product has a set of exclusive operating parameters, so that one device can be compatible with multiple different models of products.

[0080] The present invention discloses a method for detecting pits and dirt on a composite material plate based on machine vision, comprising the following specific steps:

[0081] S1: starting the visual system of the perception layer, and the visual system continuously takes pictures of the composite material plate 3;

[0082] S2: The visual program in the data and algorithm processing layer calls the image collected by the perception layer, first performs smoothing Gaussian filtering and contrast enhancement preprocessing, and then calls the YOLOX target detection model that incorporates the ECA channel attention mechanism to detect pits and dirt defects on the image;

[0083] S3: If the visual program detects that there are no defects on the composite material plate 3, the guide roller 2 in the transmission layer continues to transmit the composite material plate 3, the perception layer continues to take continuous photos of the composite material plate 3, and the visual program continues to perform defect detection. If defects are detected on the composite material plate 3, the guide roller 2 is suspended, and the transmission layer outputs the category and location information of the defect to the application layer;

[0084] S4: the labeling machine 11 in the application layer performs a labeling operation based on the information output by the transmission layer, and labels the defects at the defect positions of the composite material plate 3 according to classification;

[0085] S5: The guide roller 2 in the transmission layer continues to transport the composite material plate 3, and the perception layer continues to collect pictures. If the perception layer recognizes that there is no composite material plate 3 in the field of view, it stops collecting pictures and ends the detection. If the perception layer recognizes that there are still composite material plates 3 passing through the field of view, it continues to collect pictures and loops steps S2-S4 until all composite material plates 3 are detected.

[0086] Through continuous photography and real-time image processing, smooth Gaussian filtering and contrast enhancement preprocessing are adopted, and defect detection is performed using the YOLOX target detection model that incorporates the ECA channel attention mechanism. It is possible to quickly and accurately identify defects such as pits and dirt on the composite material plate 3. The entire process is highly automated, from image acquisition, defect detection to labeling, reducing the need for human intervention. It can not only detect the existence of defects, but also accurately determine their location and category, providing important information for subsequent processing.

[0087] This setting improves the image quality, realizes the real-time detection of the composite material plate 3 on the production line, enhances the model's attention to specific features, and improves the overall detection performance and the accuracy of detection.

[0088] Specifically, in step S1, the perception layer includes a visual system and a beam-forming optical fiber sensor 6, the visual system includes an industrial camera 7, a lens 8, a light source 9, an industrial computer 12, a light source controller 10 and a transmission cable, and the specific steps of setting the perception layer to continuously take pictures of the composite material plate 3 are as follows:

[0089] S11, determine the triggering position for taking photos: install the optical fiber sensor 6 on the frame 1, adjust the position of the optical fiber sensor 6, and ensure that the signal of the optical fiber sensor 6 is accurate when detecting that the composite material plate 3 flows through the guide roller 2;

[0090] S12, deploying camera components: the light emitting surface of the light source 9 is set at 45 degrees to the composite material plate 3 on the guide roller 2, the industrial camera 7 and the lens 8 are connected by threads, the axis of the lens 8 is perpendicular to the composite material plate 3, and the industrial camera 7, the lens 8 and the light source 9 are all fixed on the transverse bracket 4;

[0091] S13, component connection and control: the industrial camera 7 is electrically connected to the POE network port of the industrial computer 12, the light source 9 is electrically connected to the light source controller 10, the industrial computer 12, the light source controller 10, the optical fiber sensor 6 and the PLC 5 are electrically connected and signal exchange is carried out;

[0092] S14, camera component parameter setting: the light source 9 is a program-controlled stripe light, and the light source 9 parameters are set by the light source controller 10, including brightness, stripe width, stripe direction and stripe moving speed; the camera parameters are set by the industrial camera 7 imaging software NI MAX, including exposure time, contrast and Gama value;

[0093] S15, camera component imaging: when the through-beam fiber optic sensor 6 detects that a composite material plate 3 passes over the guide roller 2 at the triggering photographing position, the material presence information is sent to the PLC5 in the form of a Boolean quantity through the IO module, and the PLC5 sends a photographing signal to the industrial computer 12 through Modbus TCP. The imaging software NI MAX of LabView on the industrial computer 12 calls the industrial camera 7 driver to continuously capture images at a frequency of 15 frames per second. The latter image captured overlaps with the previous image, and there is no blind spot in the field of view. The visual program calls the image captured by the imaging software NI MAX in real time and stores the original image in a fixed folder of the industrial computer 12 according to the date and product material number.

[0094] By installing and adjusting the position of the through-beam fiber optic sensor 6 (step S11), it is ensured that the photo is triggered only when the composite material plate 3 flows through, thereby improving the pertinence and efficiency of image acquisition. By setting a specific angle between the light source 9 and the composite material plate 3 and the vertical setting of the industrial camera 7 and the lens 8, the image quality is guaranteed, which helps to improve the accuracy of subsequent defect detection. The components such as the industrial camera 7, the light source controller 10, and the through-beam fiber optic sensor 6 are electrically connected through the PLC5 and signal interaction is realized, thereby ensuring that each component can work together and improving the integration and stability of the system. According to actual needs, various parameters of the light source 9 and the industrial camera 7 are set through the material number table, so that the system can be flexibly adjusted according to different working conditions to adapt to changing detection needs. By setting the industrial camera 7 to collect pictures at a fixed frequency and ensuring that there is a certain overlap between adjacent pictures, full coverage of the surface of the composite material plate 3 is achieved, avoiding blind spots in the field of vision.

[0095] This setting can obtain high-quality original images, effectively avoid the problem of blind spots in the field of vision during detection, ensure a comprehensive evaluation of the surface condition of the composite material plate 3, adapt to different detection environments and requirements, and ensure the stability and reliability of data transmission through signal interaction between PLC5 and various components. It also facilitates later maintenance and troubleshooting. The entire process is highly automated, reducing the need for manual intervention and improving work efficiency.

[0096] Preferably, the model of the industrial camera 7 is MV-CS200-10UC, the model of the lens 8 is DTCM125-80H-AL, the model of the light source 9 is P-DIST-32ML, and the model of the optical fiber sensor 6 is FT-A11W.

[0097] Specifically, the industrial camera 7 is set to soft trigger with a frequency of 15 frames per second. After the industrial computer 12 receives the photo signal from the PLC 5, it will drive the industrial camera 7 to take continuous photos at 15 frames per second.

[0098] Preferably, the camera parameters are set by the industrial camera 7 imaging software NI MAX, and the exposure time is set to 6000us, the contrast is set to 2, and the Gama value is set to 1.5.

[0099] This setting can obtain higher quality images by accurately adjusting the exposure time, contrast and Gama value. Appropriate contrast and Gama settings can make the defect features in the image more prominent, making them easier to be recognized by the target detection algorithm, thereby improving the accuracy of detection.

[0100] Preferably, the parameters of the light source 9 are set by the light source controller 10, the brightness of the light source 9 is set to 200, the stripe width is set to 10 mm, the stripe direction is set to be from top to bottom, and the stripe moving speed is set to 100 mm / s.

[0101] The light source 9 is set to be a program-controlled stripe light, which can make the defects on the surface of the composite material plate 3 more prominent, thereby improving the defect detection accuracy and enhancing the adaptability and flexibility of the system.

[0102] Preferably, the model of the PLC5 is AM522-0808TN, the model of the industrial computer 12 is MV-IPC-F4C7Q-32G-2000G-0206-P3-X1, and the model of the light source controller 10 is PC-48W240-1TL.

[0103] The signal of the through-beam fiber optic sensor 6 is used to trigger the image acquisition of the industrial camera 7 to ensure that pictures are taken only when there is material and to avoid invalid acquisition when there is no material. The signal of the through-beam fiber optic sensor 6 is transmitted to PLC5. PLC5 coordinates the work of the industrial camera 7, lens 8, light source 9 and industrial computer 12 according to the signal to ensure that the system runs as required. When there is no material on the guide roller 2, the through-beam fiber optic sensor 6 sends a no material signal to PLC5, and the system automatically stops image acquisition and processing to avoid waste of resources.

[0104] Specifically, in step S2, the data and algorithm processing layer processes the image of the composite material plate 3 collected by the perception layer, and the specific steps are as follows:

[0105] S21, image preprocessing: after the LabView visual program obtains the original image of the composite material plate 3, the Filters filter-Gaussian function in the LabView tool software Vision Assistant is used to filter the original image, and the Contrast function is used to perform contrast enhancement on the filtered image;

[0106] S22. Load the target detection model: Load the YOLOX target detection model that incorporates the ECA channel attention mechanism in the LabView vision program;

[0107] S23, implementing defect detection: the YOLOX target detection model embedded in the LabView visual program implements defect detection on the preprocessed composite material plate 3 image, marks the defect type and confidence on the image, and outputs the pixel position coordinates of the defect;

[0108] S24, result statistics and storage: the visual program accumulates and counts the number of occurrences of each defect type and displays it on the program interface, and the pictures marked with the test results are stored in a fixed folder of the industrial computer 12 according to the date, product material number, and defect type;

[0109] Among them: When using the YOLOX algorithm to train the target detection model, a dynamic learning rate is used, and the dynamic adjustment formula is:

[0110]

[0111] Where: is the current learning rate, is the learning rate after iteration.

[0112] Through Gaussian filtering and smoothing (step S21), the noise in the image is effectively removed, and the accuracy of subsequent defect detection is improved; contrast enhancement further highlights the defect feature information in the image, which helps to identify defects more accurately. The YOLOX target detection model that incorporates the ECA channel attention mechanism is loaded and used (steps S22 and S23), which can quickly and accurately identify pits and dirt defects on the composite material plate 3, and provide detailed information such as defect type, confidence, and location coordinates. The detection results are statistically analyzed and stored (step S24), which not only facilitates real-time monitoring of production data, but also provides an important basis for subsequent quality traceability and data analysis.

[0113] This setting can effectively reduce image noise and improve image clarity, thereby ensuring the accuracy of subsequent defect detection and enabling the system to maintain high detection performance in complex backgrounds, especially for small or imperceptible defects, thereby improving the overall detection effect, helping to speed up model convergence, and improving the generalization ability and stability of the model.

[0114] Specifically, in step S4 , the industrial computer 12 is preset with a LabView visual program.

[0115] Preferably, the parameter Kernel Size of the Filters filter-Gaussian function is set to 5 to filter the original image, and the parameter of the Contrast function is set to 70 to perform contrast enhancement on the filtered image.

[0116] Specifically, in step S3, the steps of PLC5 in the transmission layer receiving the defect information sent by the data and algorithm processing layer and performing corresponding actions are as follows:

[0117] S31, detection result transmission: LabView vision program sends the defect type detected by YOLOX target detection model to PLC5 via Modbus TCP with different feedback signals;

[0118] S32, coordinate conversion and transmission: The LabView vision program converts the defect pixel position coordinates output by the YOLOX target detection model into world position coordinates after being processed by the calibration file, and sends the world position coordinates to PLC5 in the form of data volume;

[0119] S33, classified and positionally labeled: PLC5 controls multiple execution terminals of the labeling machine 11 to classify and label the defects based on the feedback signal sent by the LabView visual program and the world position coordinates of the defects. The labels are divided into two colors, yellow for repairable defects and red for unrepairable defects, and the text of the defect type is printed on each label.

[0120] Through the Modbus TCP protocol, the LabView vision program can accurately feed back the defect type identified by the YOLOX target detection model to PLC5, ensuring the real-time and accuracy of the information, converting the pixel position coordinates into world position coordinates and transmitting them to PLC5, ensuring that the labeling machine 11 can accurately mark the actual physical location and improve the positioning accuracy. According to different defect types and repairability, PLC5 controls the labeling machine 11 to classify and label the defects. This approach not only helps subsequent repair work, but also improves product quality management and traceability efficiency.

[0121] This arrangement ensures that the labeling machine 11 can label at the correct position, improves the accuracy and reliability of the operation, can promptly detect defects in the product, effectively distinguish their severity, and thus take corresponding measures, which helps to improve the overall product quality.

[0122] Specifically, in step S1, the thickness of the composite material plate 3 has various specifications, and the lens 8 is a liquid lens 8. The liquid lens 8 stores the imaging focal length by point and is bundled with the product material number to ensure that clear imaging can still be achieved when the distance from the lens 8 to the surface to be inspected of the composite material plate 3 changes.

[0123] The liquid lens 8 can quickly and accurately adjust the focal length to ensure that the camera can always capture a clear image regardless of how the thickness of the composite material board 3 changes. By binding a specific focal length to the product material number, the system can automatically identify the type of product currently being processed and quickly adjust the lens 8 to the optimal focal length, thereby supporting the detection of composite material boards 3 of various thickness specifications. There is no need to manually adjust the focal length of the lens 8, which reduces downtime on the production line and improves overall detection efficiency.

[0124] This setting enables the system to quickly adapt to composite panels of different thicknesses3, greatly enhancing the flexibility and adaptability of the system, helping to improve the accuracy of subsequent defect detection, simplifying the operator's tasks, reducing the requirements for professional skills, and also reducing the possibility of human error.

[0125] Specifically, in step S2, the YOLOX target detection model uses a generative adversarial network to perform data expansion processing on the original defect samples in the early stage of deep learning training. The generative adversarial network maximizes the probability of distinguishing real data and generated data by training the discriminator D, and trains the generator G to minimize it. The optimization function is:

[0126]

[0127] The gradient of the discriminator D is:

[0128]

[0129] The gradient of the generator G is:

[0130]

[0131] The loss functions of the generative adversarial network discriminative model and generative model are:

[0132]

[0133]

[0134] In the formula, represents the objective function; , Represent the mathematical expectations of real data x and noise data z respectively; Represents the output of the generator; , Respectively represent the output of the discriminator D corresponding to the real data and the generated data; and They represent the gradient of the discriminator D and the generator G respectively; m represents the number of samples; i and j represent the sample numbers; x and z represent the real data and noise respectively.

[0135] Finally, the two reach a Nash equilibrium, and the output of the discriminator D is 0.5, that is, the discriminator D cannot determine whether the input comes from real data or generated data, so the generated data is included in the sample set, thereby expanding the data volume of the sample set. In this embodiment, when using the generative adversarial network to derive defect samples, Adam is used as the optimizer, the momentum coefficient of the optimizer is set to 0.55, the training frequency of the generator G and the discriminator D is set to 5:1, the number of iterations n_epochs is set to 2000, the batch size batch_size is set to 16, the learning rate lr is set to 0.0001, the first-order and second-order exponential decay rates beta_1 and beta_2 of the Adam optimizer are set to 0.9 and 0.99 respectively, the image size image_size is set to 5472*3648, and the number of image channels channels is set to 3.

[0136] It should be noted that when using the YOLOX algorithm to train the target detection model, SGD is used as the optimizer, the momentum coefficient of the optimizer is set to 0.85, the first-order exponential decay rate of the optimizer beta1_1 is set to 0.99, the sample ratio of the training set, validation set and test set is 8:1:1, the batch size batch1_size is set to 8, the size of the input image image1_size is set to 5472*3648, the number of iterations n_epochs1 is set to 3000, the number of pre-training is set to 20 times, the model seed moder_seed is set to 0, the initial learning rate lr1 is set to 0.001, and the learning rate dynamic adjustment formula is:

[0137]

[0138] Where: is the current learning rate, is the learning rate after iteration.

[0139] By generating defect samples through generative adversarial networks (GANs), the data set of original defect samples can be effectively expanded and the generalization ability of the target detection model can be improved. GANs can generate high-quality synthetic images that are very close to real samples, which helps to improve the robustness of the YOLOX model when performing target detection.

[0140] This setting can enrich the number of samples without sacrificing sample quality, overcome the difficulty of obtaining enough defective samples in the early stage, help improve the training effect of the YOLOX model, and then improve its detection accuracy in practical applications.

[0141] Specifically, in step S2, the pth standard model file trained by YOLOX is first converted into an onnx file, and then LabView is used to call the onnx target detection model of YOLOX to perform defect detection on the composite material plate 3.

[0142] After conversion to the onnx format, the model can run on different frameworks and platforms. This allows the YOLOX model trained based on PyTorch to be called in LabView, facilitating deployment in different environments and reducing the need to rewrite code or adjust the model structure.

[0143] This setting enhances the flexibility of the system, allowing users to leverage existing deep learning models in different software environments without having to retrain or extensively modify existing code, thereby improving system development efficiency.

[0144] Specifically, in step S2, the function of the ECA channel attention mechanism is:

[0145]

[0146]

[0147] In the formula, are the input features, is the feature after stacking on the new dimension, is the final generated attention feature, and is the tensor average pooling function, is the tensor maximum pooling function, The convolution kernel size is The convolution operation is used to change the channel of the feature. yes Activation function, is the feature resizing function,

[0148] The size of the convolution kernel The channel dimension C can be adapted, the formula is:

[0149] .

[0150] In this embodiment , .

[0151] The ECA channel attention mechanism is integrated into the feature bottom-up feature pyramid network of the YOLOX algorithm. One-dimensional convolution is used to efficiently realize local cross-channel interaction, extract the dependency between channels, avoid adding additional network parameters, and strengthen the key feature information transmitted from the previous layer. When implementing defect detection on composite material plate 3, it can better locate and strengthen the area of ​​interest, enhance the weight of the defect position, thereby improving the detection accuracy. The ECA channel attention mechanism can also better handle defect areas of different sizes, thereby improving the detection accuracy of subtle defects.

[0152] This setting can better identify important feature areas in the image and improve the speed of target detection. The ECA channel attention mechanism is easy to integrate into the existing model and occupies very little computing power resources, so it has little impact on the processing speed of the entire network.

[0153] Specifically, in step S2, the area within the camera field of view is calibrated using Zhang's calibration method, and the pixel coordinates of the defect position are converted into world coordinates that guide the labeling machine 11 to label.

[0154] Zhang's calibration method is based on the idea of ​​homography matrix and uses the projection relationship (homography transformation) from plane to image plane to transform the calibration problem into an optimization problem. Zhang's calibration method calculates the camera's intrinsic parameters (focal length, principal point, distortion coefficient) and extrinsic parameters (rotation, translation) by taking multiple planar checkerboard images at different angles, thereby realizing the conversion from pixel coordinates to world coordinates. Through accurate coordinate conversion, it can be ensured that the labeling machine 11 can perform labeling operations at the correct physical position.

[0155] This setting can provide high-precision coordinate conversion, significantly improve the positioning accuracy of the labeling system, and complete more accurate labeling operations.

[0156] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting pits and dirt on composite material panels based on machine vision, characterized in that: The following steps are involved: S1: Start the visual system of the perception layer, and the visual system takes pictures of the composite material board continuously; S2: The visual program in the data and algorithm processing layer calls the image collected by the perception layer, first performs smoothing Gaussian filtering and contrast enhancement preprocessing, and then calls the YOLOX target detection model that incorporates the ECA channel attention mechanism to detect pits and dirt defects on the image; S3: If the visual program detects that there are no defects on the composite material plate, the guide roller in the transmission layer continues to transmit the composite material plate, the perception layer continues to take continuous photos of the composite material plate, and the visual program continues to detect defects. If defects are detected on the composite material plate, the guide roller transmission is suspended, and the transmission layer outputs the defect category and location information to the application layer; S4: The labeling machine in the application layer performs labeling based on the information output by the transmission layer, and classifies and labels the defects at the defect locations of the composite material plate; S5: The guide rollers in the transmission layer continue to transport the composite material plates, and the sensing layer continues to collect images. If the sensing layer recognizes that there are no composite material plates in the field of view, it stops collecting images and ends the detection. If the sensing layer recognizes that there are still composite material plates passing through the field of view, it continues to collect images and loops through steps S2-S4 until all composite material plates are detected.

2. The composite material plate pit and dirt detection method according to claim 1, characterized in that: In step S1, the perception layer includes a visual system and a beamforming optical fiber sensor. The visual system includes an industrial camera, a lens, a light source, an industrial computer, a light source controller, and a transmission cable. The specific steps of the perception layer for taking photos of the composite material board are as follows: S11. Determine the triggering position for taking photos: install a fiber optic sensor on the frame, adjust the position of the fiber optic sensor, and ensure that the signal is accurate when the fiber optic sensor detects that a composite material plate flows over the guide roller; S12, deploying the camera assembly: the light emitting surface of the light source is set at 45 degrees to the composite material plate on the guide roller, the industrial camera and the lens are connected by threads, the axis of the lens is perpendicular to the composite material plate, and the industrial camera, the lens and the light source are all fixed on the horizontal bracket; S13, component connection and control: the industrial camera is electrically connected to the POE network port of the industrial computer, the light source is electrically connected to the light source controller, the industrial computer, the light source controller, the optical fiber sensor and the PLC are electrically connected and the signals are exchanged; S14, camera component parameter setting: the light source is a program-controlled stripe light, and the light source parameters are set by the light source controller, including brightness, stripe width, stripe direction and stripe moving speed; the camera parameters are set by the industrial camera imaging software NI MAX, including exposure time, contrast and Gama value; S15, camera component imaging: When the through-beam fiber sensor detects that a composite material plate passes over the guide roller at the triggering position for photographing, the material information is sent to the PLC in the form of a Boolean quantity through the IO module. The PLC sends a photographing signal to the industrial computer through Modbus TCP. The imaging software NI MAX of the visual program on the industrial computer calls the industrial camera driver to capture images at a frequency of 15 frames per second. The latter image captured overlaps with the previous image, and there is no blind spot in the field of view. The visual program calls the image captured by the imaging software NI MAX in real time and stores the original image in a fixed folder of the industrial computer according to the date and product material number.

3. The composite material plate pit and dirt detection method according to claim 1, characterized in that: In step S2, the LabView language is used to write a visual program, and the data and algorithm processing layer processes the composite material board image collected by the perception layer. The specific steps are as follows: S21. Image preprocessing: After the LabView visual program obtains the original image of the composite material plate, the Filters filter-Gaussian function in the tool software Vision Assistant of the LabView visual program is used to filter the original image, and the Contrast function is used to perform contrast enhancement on the filtered image; S22. Load the target detection model: Load the YOLOX target detection model that incorporates the ECA channel attention mechanism in the LabView vision program; S23. Implement defect detection: The YOLOX target detection model embedded in the LabView visual program implements defect detection on the pre-processed composite material plate image, marks the defect type and confidence on the image, and outputs the pixel position coordinates of the defect; S24, result statistics and storage: LabView visual program accumulates and counts the number of occurrences of each defect type and displays it on the program interface. The pictures marked with the test results are stored in a fixed folder of the industrial computer according to the date, product number, and defect type. Among them: When using the YOLOX algorithm to train the target detection model, a dynamic learning rate is used, and the dynamic adjustment formula is: Where: is the current learning rate, is the learning rate after iteration.

4. The composite material plate pit and dirt detection method according to claim 3, characterized in that: In step S3, the steps of the PLC in the transmission layer receiving the defect information sent by the data and algorithm processing layer and performing corresponding actions are as follows: S31, transmission of detection results: LabView vision program sends the defect types detected by YOLOX target detection model to PLC via Modbus TCP with different feedback signals; S32, coordinate conversion and transmission: LabView vision program converts the defect pixel position coordinates output by the YOLOX target detection model into world position coordinates after being processed by the calibration file, and sends the world position coordinates to the PLC in the form of data volume; S33. Classification and position labeling: Based on the feedback signal sent by the LabView visual program and the world position coordinates of the defect, the PLC controls the multiple execution terminals of the labeling machine to classify and label the defects. The labels are divided into two colors, yellow for repairable defects and red for unrepairable defects, and the text of the defect type is printed on each label.

5. The method for detecting pits and dirt on composite material plates based on machine vision according to claim 2, characterized in that: In step S1, the thickness of the composite material board has various specifications, and the lens is a liquid lens. The liquid lens stores the imaging focal length by point and is bundled with the product part number to ensure clear imaging when the distance from the lens to the surface to be inspected of the composite material board changes.

6. The method for detecting pits and dirt on composite material plates based on machine vision according to claim 2, characterized in that: In step S1, a program-controlled stripe light source is selected, and the light source parameters are adjusted to take multiple photos of the composite material plate under different lighting effects, and then the images are synthesized.

7. The method for detecting pits and dirt on composite material plates based on machine vision according to claim 1, characterized in that: In step S2, the YOLOX target detection model uses a generative adversarial network to perform data expansion processing on the original defect samples in the early stage of deep learning training. The generative adversarial network maximizes the probability of distinguishing real data and generated data by training the discriminator D, and trains the generator G to minimize it. The optimization function is: The gradient of the discriminator D is: The gradient of the generator G is: The loss functions of the generative adversarial network discriminative model and generative model are: In the formula, represents the objective function; , Represent the real data x and the noise number respectively According to the mathematical expectation of z; Represents the output of the generator; , Respectively represent the output of the discriminator D corresponding to the real data and the generated data; and They represent the gradient of the discriminator D and the generator G respectively; m represents the number of samples; i and j represent the sample numbers; x and z represent the real data and noise respectively.

8. The method for detecting pits and dirt on composite material plates based on machine vision according to claim 3, characterized in that: In step S2, the pth standard model file trained by YOLOX is first converted into an onnx file, and then LabView calls YOLOX's onnx target detection model to perform defect detection on the composite material plate.

9. The method for detecting pits and dirt on composite material plates based on machine vision according to claim 1, characterized in that: In step S2, the function of the ECA channel attention mechanism is: In the formula, are the input features, is the feature after stacking on the new dimension, is the final generated attention feature, and is the tensor average pooling function, is the tensor maximum pooling function, The convolution kernel size is The convolution operation is used to change the channel of the feature. yes Activation function, is the feature resizing function, The size of the convolution kernel The channel dimension C can be adapted, the formula is: 。 10. The method for detecting pits and dirt on composite material plates based on machine vision according to claim 4, characterized in that: In step S3, the area within the camera field of view is calibrated using Zhang’s calibration method, and the pixel coordinates of the defect position are converted into world coordinates that guide the labeling machine to label.

Images