A method and system for image enhancement and target detection of surface defects

Through the improvement of virtual polarization filtering and YOLOV5 network, the problems of low accuracy and high cost of surface defect detection in complex lighting environments are solved, and more efficient and stable surface defect detection is achieved, which is suitable for additive manufacturing and other fields.

CN118918102BActive Publication Date: 2024-12-03NAT UNIV OF DEFENSE TECH
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411386469.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-30
Publication Date
2024-12-03
Estimated Expiration
2044-09-30

AI Technical Summary

Technical Problem

The existing surface defect detection methods have low detection accuracy and high cost in complex lighting environments, making it difficult to meet the needs of workpiece detection in additive manufacturing.

Method used

The virtual polarization filtering technology is used in combination with the YOLOV5 network to improve the robustness of image processing and object detection by building a surface defect detection model, using the DC3 module, attention mechanism and improved upsampling operator.

Benefits of technology

It significantly improves the accuracy and stability of the surface defect detection model under complex lighting conditions, reduces the detection cost, and enhances the wide applicability of the detection system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918102B_ABST
    Figure CN118918102B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for image enhancement and target detection of surface defects, the method comprising: collecting target polarization images of surface defect targets with different polarization characteristics, performing virtual polarization filtering to obtain enhanced target images; building a surface defect detection model based on the YOLOV5 network, namely: replacing the C3 module of the last layer in the backbone network and the C3 module of the middle processing layer in the neck network with the DC3 module; introducing an attention mechanism at the output end of the DC3 module in the backbone network and the neck network; improving the Upsample upsampling operator in the neck network to the Carafe upsampling operator; inputting the enhanced target image into the trained surface defect detection model to complete the defect detection of the target. The present invention is applied to the field of machine vision, and not only has high defect detection accuracy, but also has good stability and robustness.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a method and system for image enhancement and target detection of surface defects. Background Art

[0002] Additive Manufacturing technology, also known as "3D printing technology", originated in the 1960s. After decades of development, additive manufacturing technology has become an important part of aerospace, medical and automotive fields. In the AM process, a high-energy laser melts the powder layer to form a high-temperature molten pool, which then solidifies into the desired shape after cooling. Different preparation conditions will directly affect the shape of the molten pool. Improper processing parameter settings will lead to unstable molten pools, which in turn induce defects such as convex hulls, pores, and cracks. These defects will reduce the mechanical properties and molding density of the workpiece, causing the physical properties of the workpiece to deteriorate, seriously affecting the actual performance and service life of the workpiece. Therefore, it is crucial to effectively detect defective workpieces and avoid them from being put into use. Traditional non-destructive defect detection methods include radiographic testing, ultrasonic testing, liquid penetrant testing, magnetic particle testing, and eddy current testing. However, these methods are either costly, complex to operate, and slow to detect, or have specific restrictions on the objects to be detected, and have a small scope of application. Given the current expanding and flexible market, these traditional methods may not be able to fully meet production needs in some cases. Therefore, it is necessary to propose a more efficient and low-cost defect detection method.

[0003] In recent years, deep neural network technology has developed rapidly and has achieved remarkable results in the field of defect detection. In 1998, LeCun proposed LeNet, which successfully solved the handwritten digit recognition task, marking the real launch of convolutional neural network (CNN) and promoting the target detection algorithm into the deep learning stage. In 2013, Ross Girshick et al. proposed R-CNN, which applied convolutional neural network to feature extraction and improved the detection rate of data sets through the excellent feature extraction ability of CNN. In 2015, He Kaiming proposed ResNet, which broke through the limitation of neural network depth and further improved the detection accuracy. In the same year, Joseph Redmon et al. proposed YOLO, which uses a unified neural network to simultaneously predict the target position and category in the image, achieving fast and efficient target detection. In the same year, Shaoqing Ren et al. proposed Faster R-CNN, which improved the speed of training and testing stages and also improved the detection accuracy. In 2016, Gao Huang et al. proposed DenseNet, which established the connection relationship between different layers, thereby further alleviating the gradient vanishing problem and improving the accuracy of target detection. These neural network-based studies have greatly promoted the development of defect detection technology. They have improved the accuracy and processing speed of the models, allowing the detection system to more effectively identify and classify various surface defects. These studies have not only improved the performance and reliability of surface defect detection, but also expanded its scope of practical applications.

[0004] Although there are many excellent methods currently applied in the field of surface defect detection, their performance is often limited by different application scenarios when facing specific practical problems, and they fail to fully realize their potential. For example, in the detection of surface defects of workpieces, the detection accuracy of these algorithm models is usually significantly affected by complex environmental lighting. This requires replacing the corresponding detection model in different processing environments to adapt to changes in lighting, which undoubtedly increases the complexity and cost of detection. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a method and system for image enhancement and target detection of surface defects, which exhibits excellent stability and robustness in complex lighting environments and has broad application potential, helps to reduce workpiece processing costs, and promotes research progress in related fields.

[0006] To achieve the above object, the present invention provides a method for image enhancement and target detection of surface defects, comprising the following steps:

[0007] Step 1, simultaneously collecting at least three target polarization images of a surface defect target with different polarization characteristics, and performing virtual polarization filtering based on the target polarization images to obtain an enhanced target image;

[0008] The virtual polarization filtering based on the target polarization image specifically includes:

[0009] Construct a target image with the same size as each target polarization image, and calculate the Stokes vector of the incident light at the pixel point at the corresponding position on the target image based on the grayscale value of the pixel point at the same position on the three target polarization images;

[0010] Based on the Stokes vector of the incident light at each pixel point on the target image, the energy flux density of the incident light at each pixel point on the target image is calculated. ;

[0011] Calculate the signal-to-noise ratio of each pixel on the target image based on the Stokes vector:

[0012] For the target image, the signal-to-noise ratio is less than or equal to the threshold Pixels, based on the energy flux density of the incident light Calculate and obtain the initial grayscale value of the corresponding pixel;

[0013] For the target image, the signal-to-noise ratio is greater than the threshold The pixel point is superimposed with a block with an angle of After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. , and based on the energy flux density Calculate the initial grayscale value of the corresponding pixel, where: is the polarization angle of the corresponding pixel point on the target image;

[0014] Perform overexposure optimization processing on the basis of the initial grayscale value to obtain the final grayscale value of each pixel on the target image, that is, to obtain the enhanced target image;

[0015] Step 2: Build a surface defect detection model based on the YOLOV5 network, where the YOLOV5 network includes a backbone network, a neck network, and a head network. The construction process of the surface defect detection model is as follows:

[0016] The C3 module of the last layer in the backbone network and the C3 module of the middle processing layer in the neck network are replaced by the DC3 module;

[0017] At the output end of the DC3 module in the backbone network and the neck network, an attention mechanism is introduced;

[0018] The Upsample upsampling operator in the neck network is improved to a Carafe upsampling operator;

[0019] Step 3: input the enhanced target image into the trained surface defect detection model to complete target defect detection.

[0020] To achieve the above object, the present invention also provides a surface defect image enhancement and target detection system, which adopts the above surface defect image enhancement and target detection method;

[0021] The surface defect image enhancement and target detection system comprises:

[0022] An image acquisition unit, used for simultaneously acquiring at least three target polarization images of a surface defect target having different polarization characteristics;

[0023] A virtual filtering unit, used for performing virtual polarization filtering according to the target polarization image to obtain an enhanced target image;

[0024] The surface defect detection model is used to perform defect detection on the enhanced target image.

[0025] Compared with the prior art, the present invention has the following beneficial technical effects:

[0026] 1. The present invention suppresses polarized light through virtual polarization filtering, and builds a surface defect detection model based on the YOLOV5 network. The image processing technology is combined with the target detection neural network. The virtual polarization filtering significantly reduces the negative impact on target detection under complex lighting conditions, making the target area more visible, and effectively improving the target detection accuracy and stability of the surface defect detection model under complex lighting conditions, effectively overcoming the limitations of traditional methods;

[0027] 2. In the preferred embodiment of the present invention, the target pixel points of the virtual polarization filtering are screened based on the signal-to-noise ratio during the virtual polarization filtering process, thereby avoiding the suppression of the information of interest in the image and only suppressing the noise information in the image. At the same time, overexposure optimization processing is performed to avoid solving erroneous information, thereby ensuring that the enhanced target image is always better than the input target polarization image. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0029] Figure 1 Flow chart of the method for image enhancement and target detection of surface defects in an embodiment of the present invention;

[0030] Figure 2 This is a flow chart of performing virtual polarization filtering based on a target polarization image in Embodiment 1 of the present invention;

[0031] Figure 3 Schematic diagram of the structure of YOLOV5-W in Example 1 of the present invention;

[0032] Figure 4 This is a schematic diagram of the structure of the C3 module in Example 1 of the present invention;

[0033] Figure 5 This is a schematic diagram of the DC3 module structure in Example 1 of the present invention;

[0034] Figure 6 Schematic diagram of the CBAM attention mechanism structure in Example 1 of the present invention;

[0035] Figure 7 Schematic diagram of the nearest neighbor interpolation principle in Embodiment 1 of the present invention;

[0036] Figure 8 Schematic diagram of the structure of the Carafe upsampling operator in Embodiment 1 of the present invention;

[0037] Fig. 9 Schematic diagram of CIoU parameters in Embodiment 1 of the present invention;

[0038] Fig.10 This is a schematic diagram of the MPIS structure in Example 1 of the present invention;

[0039] Fig.11 is an example flow chart of image enhancement in Embodiment 1 of the present invention;

[0040] Fig.12 This is a separate schematic diagram of the virtual polarization filter input, output and grayscale histogram in Example 1 of the present invention, wherein: Fig.12 (A) is the 0° target polarization image and its grayscale histogram. Fig.12 (B) is the 45° target polarization image and its grayscale histogram. Fig.12 (C) is the 90° target polarization image and its grayscale histogram. Fig.12 (D) is the 135° target polarization image and its grayscale histogram. Fig.12 (E) is the enhanced target image and its grayscale histogram;

[0041] Fig.13 This is a schematic diagram of the accuracy of each model during the training process in Example 1 of the present invention, where: Fig.13 (A) is a schematic diagram of the accuracy of the 0° polarization direction data set. Fig.13 (B) is a schematic diagram of the accuracy of the 45° polarization direction data set. Fig.13(C) is a schematic diagram of the accuracy of the 90° polarization direction data set. Fig.13 (D) is a schematic diagram of the accuracy of the 135° polarization direction data set. Fig.13 (E) is a schematic diagram of the accuracy of the enhanced image dataset. Fig.13 (F) is a schematic diagram of average accuracy;

[0042] Fig.14 Schematic diagram of the recall rate of each model during the training process in Example 1 of the present invention, where: Fig.14 (A) is a schematic diagram of the recall rate of the 0° polarization direction dataset. Fig.14 (B) is a schematic diagram of the recall rate of the 45° polarization direction dataset. Fig.14 (C) is a schematic diagram of the recall rate of the 90° polarization direction dataset. Fig.14 (D) is a schematic diagram of the recall rate of the 135° polarization direction dataset. Fig.14 (E) is a schematic diagram of the recall rate of the enhanced image dataset. Fig.14 (F) is a diagram of average recall rate;

[0043] Fig.15 Schematic diagram of mAP0.5-0.95 of each model during the training process in Example 1 of the present invention, where: Fig.15 (A) is a schematic diagram of mAP0.5-0.95 for the 0° polarization direction dataset. Fig.15 (B) is a schematic diagram of mAP0.5-0.95 for the 45° polarization direction dataset. Fig.15 (C) is a schematic diagram of mAP0.5-0.95 for the 90° polarization direction dataset. Fig.15 (D) is a schematic diagram of mAP0.5-0.95 for the 135° polarization direction dataset. Fig.15 (E) is a schematic diagram of mAP0.5-0.95 of the enhanced image dataset. Fig.15 (F) is a schematic diagram of average mAP0.5-0.95;

[0044] Fig.16 This is a structural block diagram of the image enhancement and target detection system for surface defects in Example 2 of the present invention.

[0045] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0048] Polarization imaging technology has unique advantages over traditional optical detection methods. It can obtain spectral information, polarization information and spatial information of the detection target. It is currently widely used in industrial imaging, remote sensing, biomedical diagnosis and other fields. Defect detection based on polarization technology is conducive to extracting information such as texture structure, surface material and surface roughness from the polarization information of the detection target, which can effectively improve the accuracy and reliability of detection. Based on this, this embodiment discloses an image enhancement and target detection method for surface defects, which mainly includes several parts: first, the original image is enhanced by an image enhancement method based on polarization optical theory; second, a surface defect detection model is built on the basis of the YOLOV5 network, and the surface defect detection model is trained using the enhanced sample image to obtain a unified and widely applicable surface defect detection model; finally, the trained surface defect detection model is used to perform defect detection on the enhanced target image.

[0049] refer to Figure 1 In this embodiment, the surface defect image enhancement and target detection method specifically includes the following steps:

[0050] Step 1, simultaneously collecting at least three target polarization images of a surface defect target with different polarization characteristics, and performing virtual polarization filtering based on the target polarization images to obtain an enhanced target image;

[0051] Step 2: Build a surface defect detection model based on the YOLOV5 network. The YOLOV5 network includes a backbone network, a neck network, and a head network. The construction process of the surface defect detection model is as follows:

[0052] Replace the C3 module in the last layer of the backbone network and the C3 module in the middle processing layer of the neck network with the DC3 module;

[0053] Introduce the attention mechanism at the output of the DC3 module in the backbone network and the neck network;

[0054] Improve the Upsample upsampling operator in the neck network to the Carafe upsampling operator;

[0055] Step 3: Input the enhanced target image into the trained surface defect detection model to complete the defect detection of the target.

[0056] refer to Figure 2 In this embodiment, the virtual polarization filtering based on the target polarization image specifically includes:

[0057] Construct a target image with the same size as each target polarization image, and calculate the Stokes vector of the incident light at the pixel point at the corresponding position on the target image based on the grayscale value of the pixel point at the same position on the three target polarization images;

[0058] Based on the Stokes vector of the incident light at each pixel point on the target image, the energy flux density of the incident light at each pixel point on the target image is calculated. ;

[0059] Calculate the signal-to-noise ratio of each pixel on the target image based on the Stokes vector:

[0060] For the target image, the signal-to-noise ratio is less than or equal to the threshold Pixels, based on the energy flux density of the incident light Calculate and obtain the initial grayscale value of the corresponding pixel;

[0061] For the target image, the signal-to-noise ratio is greater than the threshold The pixel point is superimposed with a block with an angle of After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. , and based on the energy flux density Calculate the initial grayscale value of the corresponding pixel, where: is the polarization angle of the corresponding pixel point on the target image;

[0062] Based on the initial grayscale value, overexposure optimization processing is performed to obtain the final grayscale value of each pixel on the target image, that is, the enhanced target image.

[0063] In nature, light waves are electromagnetic waves, and their electric fields usually propagate in the form of plane sinusoidal waves. Therefore, for any beam of light propagating along the x-axis, its electric field It can be expressed as:

[0064] (1)

[0065] (2)

[0066] in, represents the real part, represents the electric field amplitude, represents the imaginary unit, represents the angular frequency of the electric field, represents the electric field propagation time, represents the wave number, represents the distance vector from the light source to the observation point (imaging distance vector), represents the initial phase of the electric field, represents the speed of light, represents the x-axis unit vector, is the unit vector of the electric field vibration direction, that is, the polarization direction of light;

[0067] Taking each pixel in the image as the research object, and assuming that each pixel of the image is only affected by sufficiently small points in the outside world, and is not affected by reflected light from other areas of the outside world, it can be considered that the angular frequency, propagation time, and imaging distance of the imaging light wave of each pixel are the same, so the electric field of the light wave can be expressed as a phasor form ,for:

[0068] (3)

[0069] The external environment of the electric field is idealized as a vacuum environment. From the electromagnetic field theory, we know that the electric field and its induced magnetic field Satisfies the Ampere-Maxwell law, that is:

[0070] (4)

[0071] in, represents the gradient operator, represents the vacuum permeability, represents the current density, represents the dielectric constant of vacuum;

[0072] Since it is a vacuum environment, , taking the curl on both sides of equation (4), we have:

[0073] (5)

[0074] in, represents the direction of magnetic field vibration, and solving the differential equation yields:

[0075] (6)

[0076] Therefore, the energy flux density of the electromagnetic field of the light wave is for:

[0077] (7)

[0078] The average power of the light wave is for:

[0079] (8)

[0080] in, Represents the period of a light wave.

[0081] On the other hand, starting from the morphology of photons, the relationship between the grayscale value of the pixel and the average power of the light wave is analyzed. The number of photoelectrons The functional relationship can be expressed as:

[0082] (9)

[0083] in, represents the gain of the camera, Indicates the camera bias;

[0084] The quantum efficiency of a camera is the ability to effectively convert the number of incident photons into The number of photoelectrons The relationship is described as:

[0085] (10)

[0086] in, represents the quantum efficiency function of the camera system, Indicates the wavelength of the incident light at the corresponding pixel point of the target image;

[0087] Combine equation (9) and equation (10) and substitute into the formula ,available:

[0088] (11)

[0089] in, represents the exposure time of the camera system, is Planck's constant, represents the speed of light;

[0090] Since the camera bias is often a small value, this embodiment ideally processes it to 0, and the following is obtained:

[0091] (12)

[0092] From equation (12), we can see that the grayscale value of the image is a quantity related to the wavelength of light, the average power of light, and the exposure time (there is no boundary condition between these three variables). From equation (7), we can see that while obtaining the energy flux density of light, the polarization information of the incident light is also obtained. has been eliminated, which means that even if the wavelength of the incident light, the exposure time, and the single grayscale information are known, the polarization state of the incident light cannot be inferred. Furthermore, regardless of the polarization state of the incident light, under certain conditions, the grayscale value displayed by the image is certain. Therefore, in this embodiment, the polarization state of the virtual incident light is changed by using virtual polarization filtering without affecting the original information of the image. It is theoretically feasible to enhance the image by suppressing polarized light.

[0093] In order to facilitate the description of the polarization state of light, this embodiment uses the Stokes vector The light is represented in the form of Indicates light intensity, component Represents the linear polarization component, component represents the circular polarization component, represents the transpose of the matrix. After light passes through an optical system, its polarization state changes. The Mueller matrix can describe this change. Assuming The Mueller matrix of the camera system is , then:

[0094] (13)

[0095] in, Indicates that the light passes through The Mueller matrix behind the camera system is , Indicates The elements of the Mueller matrix of the camera system;

[0096] Due to the circular polarization component of natural light Usually it is 0, so the formula (13) can be reduced to:

[0097] (14)

[0098] Taking a certain pixel point of the image obtained after imaging as the research object, we can get:

[0099] (15)

[0100] in, Indicates the power received by the camera system lens area corresponding to the pixel point. Represents the gray value of the pixel. Indicates the wavelength of the incident light corresponding to the pixel;

[0101] Therefore, the light intensity received by three camera systems with different polarization characteristics and simultaneous imaging in the same area It can be written in the following matrix form:

[0102] (16)

[0103] in, Represent the light intensity received by the three camera systems, represents a reversible matrix consisting of the values ​​of the Mueller matrix elements of multiple camera systems, Represents grayscale value The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, Represents grayscale value The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, Represents grayscale value The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image;

[0104] Since the three camera systems are imaging simultaneously, ,when When the matrix is ​​reversible, the Stokes vector of the incident light at the corresponding position of each pixel on the target image can be solved by the grayscale values ​​of the three target polarization images, that is:

[0105] (17)

[0106] in, Represents the Stokes vector of the incident light at the corresponding pixel point of the target image, Represents the grayscale value of the pixel at the same position on the three selected target polarization images;

[0107] Assuming that the exposure time of the three imaging systems is the same, we have , so this embodiment defines the energy flux density Stokes vector of the incident light at the corresponding pixel point of the target image as:

[0108] (18)

[0109] in, The Stokes vector represents the energy flux density of the incident light at the corresponding pixel point of the target image;

[0110] The polarization angle of each pixel in the target image can be obtained based on the first three items in the Stokes vector. Information, expressed as:

[0111] (19)

[0112] For a single pixel, obviously For one with Irrelevant quantity, at this time, a piece with an angle of The virtual polarizer can effectively suppress polarized light, and the reduced-dimensional Mueller matrix of the virtual polarizer is:

[0113] (20)

[0114] The energy flux density of light after passing through the virtual polarizer is:

[0115] (twenty one)

[0116] Since the wavelength of light does not change after passing through the polarizer, the pixel suppresses the initial gray value of the polarized light. for:

[0117] (twenty two)

[0118] in, represents the exposure time configured on the computer. By changing its value, the exposure of the image can be changed. From formula (22), we can see that the initial grayscale value is a Therefore, the image enhancement method in this embodiment does not require information such as the wavelength of the incident light and the exposure time, and can obtain the image according to the input target polarization image and the set exposure time ratio. By performing virtual polarization filtering, the desired target image can be obtained.

[0119] In the specific application process, if virtual polarization filtering is performed on each pixel in the target image, the information of interest in the image may be suppressed. In this embodiment, non-polarized light is determined as information and polarized light is determined as noise. Therefore, when the signal-to-noise ratio of a pixel on the target image is greater than the threshold When the signal-to-noise ratio is greater than the threshold, it can be considered that the pixel point is richer than the noise, and no virtual polarization filtering is required. Therefore, in the specific implementation process, only the pixel point with a signal-to-noise ratio greater than the threshold is Virtual polarization filtering is performed on the pixels of the target image, while other pixels are not. That is, the initial grayscale value of each pixel on the target image is:

[0120] (twenty three)

[0121] (twenty four)

[0122] in, Indicates the signal-to-noise ratio of the corresponding pixel in the target image.

[0123] In the above calculation process, both in the calculation of the Stokes vector part and in the update of the pixel gray value part, it depends on formula (12). However, formula (12) is only valid when the image is not overexposed. When the input image is overexposed, erroneous information may be generated, thereby reducing the imaging effect and affecting the accuracy of subsequent target detection. Since it is difficult to avoid the overexposure of the image under complex lighting conditions, this embodiment performs overexposure optimization processing to ensure that the enhanced target image is always better than the input target polarization image.

[0124] Since the circularly polarized light component in nature is usually zero, the light wave can be regarded as a combination of unpolarized light and linearly polarized light. The relationship between its reduced-dimensional Stokes vector can be expressed as:

[0125] (25)

[0126] in, is the reduced-dimensional Stokes vector of unpolarized light, is the reduced-dimensional Stokes vector of polarized light;

[0127] The polarization angle of the linear polarization part is recorded as , which can be obtained using formula (19); let the angle of the virtual polarizer be , then the light intensity passing through the virtual polarizer is for:

[0128] (26)

[0129] If the input image is overexposed, there are two cases for discussion below:

[0130] like ,but : At this time, all the input images must be overexposed, and the method of this embodiment cannot calculate any information from them. The initial grayscale value of the output is 255;

[0131] like Not much greater than : At this time, the input image may contain an image that is not overexposed. If so, the initial grayscale value calculated by the method of this embodiment is Must be greater than the gray value without overexposure The product of the exposure time ratio does not play a filtering role, so we can make , to achieve overexposure processing.

[0132] Therefore, the process of overexposure optimization processing in this embodiment is specifically as follows:

[0133] (27)

[0134] in, Indicates the final grayscale value of the corresponding pixel of the target image;

[0135] After calculating the final grayscale value of each pixel of the target image, the final target image is obtained, and the image enhancement operation is completed.

[0136] As a preferred implementation, when the number of camera systems with different polarization characteristics and that simultaneously image is four or more, that is, when the number of target polarization images is greater than 3, the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated based on the grayscale values ​​of the pixel points at the same position on the three target polarization images. Specifically, the three smallest values ​​are selected from all grayscale values ​​at the corresponding positions on all target polarization images, and the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated to avoid selecting overexposed pixel grayscale values ​​as much as possible.

[0137] YOLO (You Only Look Once) is a class of high-performance real-time target detection algorithms, known for its single-stage detector design, which can achieve real-time inference speed while maintaining high detection accuracy. The YOLO algorithm regards the target detection problem as a regression task, and effectively performs target detection by directly predicting the bounding box position and category probability in a single neural network. This design makes the YOLO algorithm the first choice for many researchers and engineers. At present, the YOLO series of algorithms has developed to the 10th generation, among which the YOLOV5 network has been optimized in many aspects, including the adaptive anchor mechanism and advanced feature extraction technology, thereby significantly improving the detection accuracy. The YOLOV5 network has shown excellent performance in various application scenarios, demonstrating its high efficiency and accuracy in detection tasks. Therefore, this embodiment selects the YOLOV5 network as the basic template of the surface defect detection model, and improves it on this basis to improve the accuracy and robustness of defect detection after IEVPF (enhanced image edge processing and feature fusion). The improvement aims to utilize the efficient characteristics of the YOLOV5 network and the enhancement effect of the IEVPF technology to further optimize the performance of the defect detection system.

[0138] In this embodiment, in order to improve the actual performance of the YOLOV5 network in the surface defect detection task, this embodiment makes four improvements to the YOLOV5 network 5 to form a surface defect detection model named YOLOV5-W. The specific improvement measures are as follows:

[0139] 1. The C3 (CSP Bottleneck with 3convolutions) module in the last layer of the backbone network of the YOLOV5 network and the C3 module in the middle processing layer of the neck network are replaced with the DC3 (DenseNet-C3) module to enhance the model's capabilities in deep feature extraction and feature fusion. This change utilizes the dense connection characteristics of DenseNet and effectively improves the efficiency of information flow and gradient propagation;

[0140] 2. The CBAM (Convolutional Block Attention Module) attention mechanism is introduced at the output end of the DC3 module in the backbone network and the neck network to improve the expressiveness of the feature map and enhance the model's ability to detect small targets;

[0141] 3. The Upsample upsampling operator in the neck network is improved to the Carafe upsampling operator to improve the meticulousness and accuracy of the upsampling results and improve the quality of detail recovery;

[0142] 4. The EIOU loss function is used instead of the CIOU loss function. The improved bounding box regression mechanism improves the convergence speed and optimization ability of the model, while enhancing the detection performance of small targets.

[0143] In the specific implementation process, the input size of the surface defect detection model is 640×640×3. In order to adapt to the input size of 640×640×3, this embodiment copies the pixel values ​​of the single-channel target image to all three color channels. Figure 3In the backbone network, the CSPDarkNet53 structure is adopted. The shallow features are extracted and fused by combining the CBS module and the C3 module. Then, the DC3 module is used to extract the deep features and fuse them with the shallow features. The CBAM module and the SPPF module further improve the feature expression ability, optimize the computational efficiency and enhance the robustness of the model. In the neck network, the feature pyramid network (PANet) structure is adopted. The network architecture is deepened through the top-down and bottom-up feature fusion mechanism, multi-scale features are integrated, and the semantic representation of defects is enhanced through feature maps of three different scales. In order to process the weak defect features in the image, the DC3 and CBAM modules are introduced in the shallow layer of the neck network in this embodiment to achieve more refined semantic feature extraction, enhance the feature expression ability, and effectively suppress background noise. Finally, the head network extracts feature maps of three scales of 80×80×128, 40×40×256 and 20×20×512 from the neck network, and generates the final output vector containing the category probability, confidence score and bounding box during the inference process. The surface defect detection model in this embodiment can significantly enhance the ability to extract deep features without significantly increasing the model complexity and computational cost, thereby effectively improving the performance of surface defect detection.

[0144] The C3 module in the original YOLO-V5 model is based on the ResNet architecture, and its feature information propagation expression is:

[0145] (28)

[0146] in, represents the nonlinear transfer function, , Respectively The input of the layer.

[0147] The C3 module constructed in this way is as follows Figure 4 As shown, the C3 module enhances the feature extraction and fusion capabilities of the network through residual connections, thereby improving the accuracy of the model and the object detection performance. However, since the C3 module has fewer layers and the intermediate layers are not directly connected to the output layer, this may lead to incomplete extraction of feature semantic information and may lose some key information of the intermediate layers. To solve this problem, this embodiment introduces the DC3 module built on DenseNet. The feature information propagation mechanism of DenseNet directly connects the feature maps of all previous layers, allowing each layer to access the features of all previous layers, thereby enhancing the continuity of feature transfer and the expressiveness of the network. Specifically, the feature propagation expression of DenseNet is:

[0148] (29)

[0149] in, is the transfer function.

[0150] The DC3 module constructed in this way is as follows Figure 5 As shown in the figure, the DC3 module draws on the dense connection mechanism of DenseNet, and directly connects the feature map of each layer with the feature maps of all previous layers, which promotes the reuse of features and the effective propagation of gradients. This design not only alleviates the gradient disappearance problem, but also improves the utilization efficiency of feature information, further improving the performance in detection tasks.

[0151] In surface defect images, defect areas usually occupy fewer pixels, belong to the category of small target detection, and are easily affected by background and other interference factors. In order to overcome the problem of lack of attention mechanism in the original network, this embodiment introduces the Convolutional Block Attention Module (CBAM) attention mechanism in the backbone network and the neck network to enhance the network's ability to focus on key targets. The CBAM attention mechanism introduces two attention mechanisms to dynamically adjust the weights of feature maps, thereby effectively improving the ability to identify small targets and tiny features. The structure of this mechanism is as follows: Figure 6 As shown. Figure 6 It can be seen that the attention mechanism in this embodiment includes two independent sub-modules, namely CAM (Channel Attention Module) and SAM (Spartial Attention Module), which are attention mechanisms acting on channels and spaces respectively.

[0152] In the specific implementation process, the process of the channel attention mechanism CAM is as follows: the input feature map F (H × W × C) is subjected to global max pooling and global average pooling respectively to obtain two feature maps of size 1 × 1 × C; the two feature maps of size 1 × 1 × C are respectively used as the input of a two-layer neural network MLP to obtain the corresponding two outputs, and then the two outputs are subjected to Elelment-Wise addition operation, and then subjected to sigmoid activation operation to generate channel attention features , channel attention feature It is used to dynamically adjust the importance of each channel in the feature map, thereby improving the network's responsiveness to key features. Specifically:

[0153] (30)

[0154] in, is the sigmoid activation function, are the parameters of the neural network MLP, is the output feature map of global average pooling, is the output feature map of the global maximum pooling, H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map.

[0155] The process of the spatial attention mechanism is: the input feature map of the channel attention mechanism With channel attention features Perform element-by-element multiplication to generate the input feature map of the spatial attention mechanism ; Input feature map After global maximum pooling and global average pooling, two feature maps of size H×W×1 are obtained. The two feature maps of size H×W×1 are concatenated and Convolution operation is performed to generate spatial attention feature map, and then sigmod is used to generate spatial channel features. , spatial channel characteristics By dynamically adjusting the spatial importance of the feature map, the network's sensitivity to key spatial areas is improved. Specifically:

[0156] (31)

[0157] in, The convolution kernel size is Convolution operation;

[0158] Finally, the input feature map of the spatial attention mechanism and spatial channel characteristics Perform element-by-element multiplication to generate feature output optimized by the attention mechanism.

[0159] The attention mechanism module in this embodiment uses a layer-by-layer attention mechanism to strengthen the channel dimension and spatial dimension of the feature map, thereby achieving more accurate feature modeling and processing in the feature extraction process, which can effectively improve the robustness of the model. At the same time, due to its simple design and low computational overhead, it has high efficiency in practical applications and helps to improve the detection and classification capabilities of the overall model.

[0160] The upsampling method used in the neck network of the original YOLOV5 model is the nearest neighbor interpolation. The basic principle of this method is to set the grayscale value of the transformed pixel to the grayscale value of the input pixel closest to the pixel. Specifically, during the upsampling process, the nearest neighbor interpolation fills the pixels of the output image by selecting the pixel value closest to the target position in the input image, thereby simplifying the interpolation calculation and retaining the original features, such as Figure 7However, this method may result in jagged edges and loss of details in the reconstructed image, thus affecting the accuracy of defect detection. Therefore, in terms of feature recovery and image reconstruction accuracy, more complex interpolation methods are often required. To this end, this embodiment introduces the Carafe upsampling operator in the model. The principle is as follows: Figure 8 As shown in the figure, the Carafe operator has a large receptive field and can effectively aggregate the global information of the image. The operator supports content-aware processing for specific instances and enhances the expressiveness of feature maps by dynamically generating adaptive convolution kernels. Compared with traditional methods, the Carafe operator performs better in terms of computational overhead, with lower computational cost, smaller model size, and higher computational efficiency. This feature enables Carafe to maintain a faster processing speed while improving image reconstruction and feature extraction performance.

[0161] The Carafe upsampling operator has two main modules: the upsampling kernel prediction module and the feature recombination module:

[0162] The upsampling kernel prediction module first performs channel compression on the feature map, converting the size of the original feature map from H×W×C to H×W×C m , where C m is the number of channels after compression, which is implemented using a 1×1 convolution operation. Next, further convolution operations are performed on the compressed feature map to generate H×W×σ 2 k up 2 Here, σ represents the upsampling scale factor, k up is the size of the upsampling kernel. Finally, this feature map is expanded to a shape of σH×σW×k up 2 To ensure the stability of the convolution operation, the upsampling kernel is normalized so that the sum of its weights is 1, thus keeping the overall energy and information content of the feature map unchanged;

[0163] The feature reconstruction module then applies the upsampling kernel to the input feature map to map the information in the low-resolution feature map to the high-resolution space. Specifically, the process uses the predicted upsampling kernel to perform point-by-point weighted summation at each location to generate a high-quality high-resolution feature map. This method effectively restores image details and improves image clarity through careful weighted operations, thereby optimizing the spatial resolution and feature representation of the image.

[0164] The accuracy, training efficiency and robustness of the target detection model are greatly affected by the choice of loss function. Choosing an appropriate loss function can significantly improve the overall performance of the model. In this embodiment, the loss function of the surface defect detection model during training is composed of the weighted sum of three parts: position loss, classification loss and confidence loss. The original YOLOV5 model uses the position loss CIoU (Complete Intersection over Union) as the loss function. CIoU introduces the aspect ratio measurement term between the predicted box and the true box based on DIoU (Distance Intersection over Union), thereby effectively speeding up the regression speed of the predicted box and improving the training efficiency. Its mathematical expression is:

[0165] (32)

[0166] (33)

[0167] (34)

[0168] in, is the intermediate parameter, is the ratio of the area of ​​the intersection of the predicted box and the true box to the area of ​​the union. is the Euclidean distance, is the diagonal distance of the minimum enclosing rectangle, are the center points of the target box and the prediction box respectively. are the predicted box width and height respectively, are the width and height of the real frame respectively; the physical meaning of each parameter is as follows Fig. 9 shown.

[0169] Although the CIoU loss function performs well in the box regression task, it still has some significant limitations. For example, when the aspect ratio of the predicted box is linearly proportional to the true box, the relative proportion penalty term in the CIoU loss function may fail, which causes the loss function to be unable to effectively guide the optimization process in this case. In addition, according to the gradient formula for the width and height of the predicted box, when the value of one dimension increases, the value of the other dimension must decrease accordingly. This feature limits the ability of the box size to increase or decrease simultaneously during the regression process. Therefore, when dealing with scenes with inconsistent aspect ratios, the CIoU loss may not be able to fully optimize the shape and position of the box, thereby affecting the detection accuracy and generalization ability of the model. To this end, this embodiment introduces the EIoU (Enhanced Intersection over Union) loss to replace the CIoU loss. Its expression is as follows:

[0170] (35)

[0171] in, is the IoU loss, is the distance loss, is the direction loss, The width and height of the smallest box that covers the predicted box and the target box, respectively. The EIOU loss function significantly improves the accuracy and convergence speed of box regression by directly minimizing the difference in width and height between the predicted box and the true box. This loss function not only focuses on the overlapping area between the predicted box and the true box, but also considers the direct difference in the width and height of the box, thereby providing more refined gradient information during the optimization process. It enables the model to adjust the size of the box faster and reduce the geometric difference between the predicted box and the true box, thereby improving the accuracy of target detection and the training efficiency of the model. In this way, the EIOU loss function shows superior performance and higher robustness when dealing with various target scales and shapes.

[0172] It is worth noting that although this embodiment Figure 1 , Figure 2 The steps in the process are shown in sequence as indicated by the arrows, but the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of the steps, and the steps can be executed in other orders. Figure 1 , Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0173] The image enhancement and target detection method of surface defects in this embodiment is further explained below in conjunction with a specific experimental example, wherein the experimental example is divided into two parts: a virtual polarization filtering image enhancement experiment and a surface defect detection experiment.

[0174] In the virtual polarization filter image enhancement experiment, the Fig.10The multi-source polarization imaging system (MPIS) shown in the figure collects image data. The imaging system mainly includes the workpiece to be measured, LED light source, linear polarizer, electric rotating frame, CMOS camera and computer. Among them, the linear polarizer uses THORLABS's LPVISE100-A linear polarizer, which has a size of 25.4mm and an operating wavelength range of 400-700nm; the resolution of the CMOS image sensor is 2448×2048, the single pixel size is 3.45μm, and the lens focal length is 50mm; the electric rotating frame uses Standa's FPSTA-8MPR16-1, which can achieve 360° rotation angle and 0.75arcmin step resolution polarizer rotation control, and its equipped controller is 8SMC4-USB. The computer model used for image acquisition and analysis in the experimental example is ThinkPad S2, the CPU is i5 8250U, and its maximum frequency is 1.8G Hz. During the experimental example, the workpiece to be measured is illuminated by an LED light source, and the light is reflected by the workpiece surface and then collected by the CMOS image sensor through the polarizer.

[0175] refer to Fig.11 In the virtual polarization filter image enhancement experiment, the main tasks include data acquisition, image sampling, input determination, and solution. , Solve And output results in six steps. Specifically, the detailed method of each step is as follows:

[0176] 1. MPIS collects images of defective objects and obtains four polarization images of the target at different directions of 0°, 90°, 45° and 135° at the same viewing angle;

[0177] 2. The computer receives four target polarization images as input and samples them using the opencv library to obtain the corresponding grayscale value matrix;

[0178] 3. Select the pixel points to be processed on the target image, and select the three with the smallest values ​​from the four gray value matrices to form the input;

[0179] 4. Determine the matrix based on the three grayscale values ​​input W And solve ;

[0180] 5. According to the principle of the method of this embodiment, calculate the new final gray value of the pixel ;

[0181] 6. If all pixels in the target image have been processed, the enhanced target image is output; otherwise, return to step 3.

[0182] The results of the virtual polarization filtering image enhancement experiment are evaluated by analyzing the grayscale histogram of the image. The grayscale histogram can effectively display the pixel frequency of different grayscale levels in the image, which helps to analyze the contrast, brightness and dynamic range of the image. In this example, according to the principle of the method of this embodiment, most of the pixels in the image are subjected to virtual polarization filtering in the optimal polarization direction. The expected result is that most of the low and medium grayscale value areas in the grayscale histogram of the ideal output image will show a downward trend. This feature shows that the grayscale value of the low contrast area is reduced by virtual polarization filtering, thereby suppressing the background expression and improving the visual quality and feature performance of the image.

[0183] refer to Fig.12 , are the experimental results and histogram of image enhancement based on virtual polarization filtering in this example, and Table 1 shows the distribution of each gray value interval of each image.

[0184] Table 1

[0185]

[0186] Depend on Fig.12 As shown in Table 1, after image enhancement, the number of pixels with grayscale values ​​greater than 204 is reduced by two orders of magnitude compared with the images with polarization directions of 0°, 45°, and 90°, showing a significant effect of suppressing overexposure. Compared with the image with polarization direction of 135°, the number of pixels with grayscale values ​​greater than 152 has changed little, only decreasing by about 7%. However, in the low grayscale value area, the number of pixels with grayscale values ​​in the range of [0, 50] increased by 33%, while the number of pixels with grayscale values ​​in the range of [51, 101] decreased by 25%. This shows that the enhanced image maintains relatively stable features in the high grayscale value area, while the overall grayscale level in the low grayscale value area is reduced, thereby effectively suppressing background noise. This processing method helps to improve the sensitivity and accuracy of the model in defect detection tasks, because the reduction of background noise can highlight the characteristics of the target area and make the identification of defects clearer.

[0187] In the surface defect detection experiment part, accuracy, recall rate and mAP value are used as evaluation criteria to verify the actual application effect of the improved model in this embodiment.

[0188] Accuracy P It refers to the proportion of actual correct target instances among all instances detected as targets, which is:

[0189] (36)

[0190] in, TPrepresents True Positives, that is, the number of correctly detected targets, FP Represents false positives, that is, the number of non-target instances that are mistakenly detected as targets. P It is used to measure the accuracy of the detection system, that is, the proportion of samples marked as targets that the system actually correctly identifies. Accuracy means that the false alarm rate of the detection system is low, which can effectively reduce false target identification and improve the reliability of overall detection.

[0191] Recall R It refers to the proportion of samples that are correctly identified as positive among all actual positive samples, which is:

[0192] (37)

[0193] in, FN Represents false negatives, that is, the number of samples that are actually positive but are mistakenly identified as negative. Recall rate measures the ability of the detection system to identify all true positive samples. A high recall rate means that the system has a lower proportion of missed detections in actual positive samples, thus enabling more comprehensive detection of targets and improving detection integrity.

[0194] mAP is calculated by averaging the average precision (AP) of all categories to evaluate the overall performance of the model on multiple categories:

[0195] (38)

[0196] in, Indicates The average precision of the categories, is the number of categories. The average precision (AP) is obtained by calculating the comprehensive performance of precision and recall of each category, usually including the area under the precision-recall curve. The calculation of mAP can comprehensively evaluate the detection ability of the model in multiple categories and provide a comprehensive indicator for the performance of the model. Especially when dealing with multi-category problems, it can effectively reflect the overall effect of the model.

[0197] Table 2 shows the hyperparameter settings for the surface defect detection experiment.

[0198] Table 2

[0199]

[0200] The Loss values ​​of the existing YOLOV5 model and the YOLOV5-W model in this embodiment trained on the data set X are averaged after 50 iterations, as shown in Table 3.

[0201] Table 3

[0202]

[0203] In the 0° polarization direction dataset, 45° polarization direction dataset, 90° polarization direction dataset, 135° polarization direction dataset, and enhanced image dataset shown in Table 3, the Loss values ​​are reduced by 51.1%, 34.5%, 50.6%, 32.0%, and 33.1%, respectively, and the average reduction is 40.3%. It can be seen that no matter which dataset is used, YOLOV5-W has shown superior performance compared to YOLOV5. Compared with the model trained on the four polarization direction datasets, the Loss values ​​of the YOLOV5 model trained on the dataset enhanced by virtual polarization filtering are reduced by 37.2%, 24.7%, 31.0%, and 12.6%, respectively, and the average reduction is 26.4%. Compared with the model trained on the four polarization direction datasets, the Loss values ​​of the YOLO V5-W model trained on the dataset after virtual polarization filtering are reduced by 14.0%, 23.1%, 6.7%, and 14.0%, respectively, and the average reduction is 14.5%. These results not only show that the dataset after virtual polarization filtering has better applicability for object detection tasks, but also show the strong robustness of the YOLO V5-W model and its ability to effectively adapt to changes in the dataset.

[0204] refer to Fig.13 This is a schematic diagram of the accuracy during the training process. Fig.13 (A)- Fig.13 It can be observed in (E) that the YOLOV5-W model shows higher accuracy than the YOLOV5 model during training. Fig.13 The "origin" curve in (F) represents the average accuracy of the model trained on datasets with four different polarization directions, namely Fig.13 (A)- Fig.13 (D) is the average of the eight curves; the "enhance" curve represents the average accuracy of the model trained on the dataset enhanced by virtual polarization filtering, i.e. Fig.13 (E) The average of the two curves. These results show that the dataset enhanced with virtual polarization filtering and the improved YOLOV5-W model both improve the accuracy of the model.

[0205] The average accuracy of each model after 50 iterations is taken, as shown in Table 4.

[0206] Table 4

[0207]

[0208] The training results on the 0° polarization direction dataset, 45° polarization direction dataset, 90° polarization direction dataset, 135° polarization direction dataset, and enhanced image dataset show that the accuracy of the YOLOV5-W model increased by 11.8%, 14.6%, 7.3%, 9.3%, and 11.2%, respectively, with an average increase of 10.8%. This result shows that YOLOV5-W has higher accuracy than the YOLO V5 model on all datasets. Compared with the model trained on the four polarization direction datasets, the accuracy of the YOLOV5 model trained on the dataset enhanced by virtual polarization filtering increased by 3.0%, 7.1%, -2.4%, and 3.4%, respectively, with an average increase of 2.8%. Compared with the model trained on the four polarization direction datasets, the accuracy of the YOLOV5-W model trained on the dataset enhanced by virtual polarization filtering increased by 2.4%, 3.7%, 1.52%, and 5.33%, respectively, with an average increase of 3.2%. This shows that training the model using a dataset enhanced by virtual filtering can improve the recognition accuracy of the model, supporting the potential of virtual filtering as an effective data enhancement technology in improving the performance of surface defect detection models.

[0209] refer to Fig.14 This is a schematic diagram of the recall rate during the training process. Fig.14 (A)- Fig.14 It can be observed in (E) that the recall of YOLOV5-W is higher than that of YOLOV5 during training. Fig.14 (F) shows that training the model with the dataset enhanced by virtual polarization filtering can slightly improve the recall rate of the model. These results show that both the dataset enhanced by virtual polarization filtering and the improved YOLO V5-W model improve the recall rate of the model.

[0210] The average accuracy of each model after 50 iterations is taken, as shown in Table 5.

[0211] Table 5

[0212]

[0213] On the 0° polarization direction dataset, 45° polarization direction dataset, 90° polarization direction dataset, 135° polarization direction dataset, and enhanced image dataset, the recall rate increased by 10.8%, 16.0%, 7.9%, 6.7%, and 10.2%, respectively, with an average increase of 10.3%. It can be seen that YOLO V5-W has a higher recall rate than YOLO V5 on any dataset. Compared with the model trained on the four polarization direction datasets, the recall rate of the YOLOV5 model trained on the dataset enhanced by virtual polarization filtering increased by 2.4%, 10.8%, -1.0%, and 2.0%, respectively, with an average increase of 3.6%. Compared with the model trained on the four polarization direction datasets, the recall rate of the YOLO V5-W model trained on the dataset enhanced by virtual polarization filtering increased by 1.7%, 4.9%, 1.3%, and 5.4%, respectively, with an average increase of 3.3%. This shows that training a model using a dataset enhanced with virtual filtering can improve the recall rate of the model, supporting the potential of virtual filtering as an effective data enhancement technology in improving the performance of surface defect detection models.

[0214] refer to Fig.15 This is a schematic diagram of mAP during training. Fig.15 (A)- Fig.15 It can be observed in (E) that the mAP value of YOLOV5-W is higher than that of YOLOV5 during training. Fig.15 (F) shows that training the model with the dataset enhanced by virtual polarization filtering can enhance the mAP value of the model. These results show that both the dataset enhanced by virtual polarization filtering and the improved YOLO V5-W model improve the mAP value of the model.

[0215] The average accuracy of each model after 50 iterations is taken, as shown in Table 6.

[0216] Table 6

[0217]

[0218] On the 0° polarization direction dataset, 45° polarization direction dataset, 90° polarization direction dataset, 135° polarization direction dataset, and enhanced image dataset, the mAP values ​​increased by 14.9%, 15.0%, 13.2%, 12.6%, and 12.7%, respectively, with an average increase of 13.7%. It can be seen that YOLO V5-W has a higher mAP value than YOLO V5 on any dataset. Compared with the model trained on the four polarization direction datasets, the mAP values ​​of the YOLO V5 model trained on the dataset enhanced by virtual polarization filtering increased by 6.4%, 11.6%, 2.4%, and 6.3%, respectively, with an average increase of 6.7%. Compared with the model trained on the four polarization direction datasets, the mAP values ​​of the YOLO V5-W model trained on the dataset enhanced by virtual polarization filtering increased by 4.2%, 9.3%, 1.7%, and 6.4%, respectively, with an average increase of 5.4%. This shows that training the model using the dataset enhanced by virtual filtering can slightly improve the mAP value of the model, supporting the potential of virtual filtering as an effective data enhancement technology in improving the performance of surface defect detection models.

[0219] from Figures 13 to 15 It can be seen that:

[0220] 1. For the same model, the recognition accuracy of the model trained on the dataset enhanced by virtual polarization filtering is higher than that of the model trained on the dataset of four polarization filtering directions;

[0221] 2. For the same data set, the recognition accuracy of YOLO V5-W is higher than that of YOLO V5;

[0222] 3. In general, YOLO V5-W-Enh, the improved YOLO V5-W model trained on the virtual polarization filter dataset, is the only one that has no false detection or missed detection in the image experiment after virtual polarization filter enhancement, showing the highest recognition accuracy and confidence;

[0223] 4. When the model trained on the virtual polarization filter enhanced dataset detects images with poor polarization suppression effects (0° and 90° in the figure), it will mistakenly identify some continuous convex hulls as scratches. This is because there is a big difference between scratches and continuous multiple convex hulls in images enhanced by virtual polarization filtering, while in images with poor polarization suppression effects, the difference between the two is small. Therefore, it can be shown that images enhanced by virtual polarization filtering are more conducive to the training of the model for target detection;

[0224] These four points illustrate that the virtual polarization filtering method and the improved YOLOV5-W model proposed in this embodiment can effectively enhance the accuracy of surface defect recognition.

[0225] Example 2

[0226] Based on the image enhancement and target detection method for surface defects in Example 1, this embodiment discloses an image enhancement and target detection system for surface defects, referring to Fig.16 The system includes an image acquisition unit, a virtual filtering unit and a surface defect detection model, specifically:

[0227] The image acquisition unit is used to simultaneously acquire at least three target polarization images of the surface defect target with different polarization characteristics;

[0228] The virtual filtering unit is used to perform virtual polarization filtering according to the target polarization image to obtain an enhanced target image;

[0229] The surface defect detection model is used to perform defect detection on the enhanced target image.

[0230] In this embodiment, the specific working process and working principle of the image acquisition unit, the virtual filtering unit and the surface defect detection model are the same as those in the method of Embodiment 1, so they are not described in detail in this embodiment. Each unit module can be implemented in whole or in part by software, hardware and a combination thereof, and each unit module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above unit modules.

[0231] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A method for image enhancement and target detection of surface defects, It is characterized in that The steps include: Step 1, simultaneously collecting at least three target polarization images of a surface defect target with different polarization characteristics, and performing virtual polarization filtering based on the target polarization images to obtain an enhanced target image; The virtual polarization filtering based on the target polarization image specifically includes: Construct a target image with the same size as each target polarization image, and calculate the Stokes vector of the incident light at the pixel point at the corresponding position on the target image based on the grayscale value of the pixel point at the same position on the three target polarization images; Based on the Stokes vector of the incident light at each pixel point on the target image, the energy flux density of the incident light at each pixel point on the target image is calculated. ; Calculate the signal-to-noise ratio of each pixel on the target image based on the Stokes vector: For the target image, the signal-to-noise ratio is less than or equal to the threshold Pixels, based on the energy flux density of the incident light Calculate and obtain the initial grayscale value of the corresponding pixel; For the target image, the signal-to-noise ratio is greater than the threshold The pixel point is superimposed with a block with an angle of After the virtual polarizer is inserted, the energy flux density of the light passing through the virtual polarizer is obtained. , and based on the energy flux density Calculate the initial grayscale value of the corresponding pixel, where: is the polarization angle of the corresponding pixel point on the target image; Perform overexposure optimization processing on the basis of the initial grayscale value to obtain the final grayscale value of each pixel on the target image, that is, to obtain the enhanced target image; Step 2: Build a surface defect detection model based on the YOLOV5 network, where the YOLOV5 network includes a backbone network, a neck network, and a head network. The construction process of the surface defect detection model is as follows: The C3 module of the last layer in the backbone network and the C3 module of the middle processing layer in the neck network are replaced by the DC3 module; At the output end of the DC3 module in the backbone network and the neck network, an attention mechanism is introduced; The Upsample upsampling operator in the neck network is improved to a Carafe upsampling operator; Step 3: input the enhanced target image into the trained surface defect detection model to complete target defect detection.

2. The method for image enhancement and target detection of surface defects according to claim 1, It is characterized in that In step 2, the attention mechanism includes a channel attention mechanism and a spatial attention mechanism; The process of the channel attention mechanism is as follows: the input feature map F (H × W × C) is subjected to global maximum pooling and global average pooling respectively to obtain two feature maps of size 1 × 1 × C; the two feature maps of size 1 × 1 × C are respectively used as the input of a two-layer neural network MLP to obtain two corresponding outputs, and then the two outputs are subjected to Elelment-Wise addition operation, and then subjected to sigmoid activation operation to generate channel attention features ,Right now: ; in, is the sigmoid activation function, are the parameters of the neural network MLP, is the output feature map of global average pooling, is the output feature map of the global maximum pooling, H is the height of the feature map, W is the width of the feature map, and C is the number of channels of the feature map; The process of the spatial attention mechanism is as follows: the input feature map of the channel attention mechanism With channel attention features Perform element-by-element multiplication to generate the input feature map of the spatial attention mechanism ; Input feature map After global maximum pooling and global average pooling, two feature maps of size H×W×1 are obtained. The two feature maps of size H×W×1 are concatenated and Convolution operation is performed to generate spatial attention feature map, and then sigmod is used to generate spatial channel features. ,for: ; in, The convolution kernel size is Convolution operation; Finally, the input feature map of the spatial attention mechanism is and spatial channel characteristics Perform element-by-element multiplication to generate feature output optimized by the attention mechanism.

3. The method for image enhancement and target detection of surface defects according to claim 1, It is characterized in that In step 2, the loss function of the surface defect detection model during training is composed of the weighted sum of position loss, classification loss and confidence loss, where position loss The calculation process is: ; in, is the IoU loss, is the distance loss, is the direction loss, is the ratio of the area of ​​the intersection of the predicted box and the true box to the area of ​​the union. is the Euclidean distance, are the predicted box width and height respectively, , are the width and height of the real frame respectively, The width and height of the smallest box that covers the prediction box and the target box respectively.

4. The method for image enhancement and target detection of surface defects according to claim 1, 2 or 3, It is characterized in that When the number of target polarization images is greater than 3, the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated based on the grayscale value of the pixel point at the same position on the three target polarization images, specifically: The three smallest values ​​are selected from all grayscale values ​​at corresponding positions on all target polarization images, and the Stokes vector of the incident light at the pixel point at the corresponding position on the target image is calculated.

5. The method for image enhancement and target detection of surface defects according to claim 4, It is characterized in that The Stokes vector of the incident light at each pixel point on the target image is specifically: ; in, Represents the Stokes vector of the incident light at the corresponding pixel point of the target image, Represents the power of the incident light at the corresponding pixel point of the target image, represents the gain of the camera system, is Planck's constant, represents the speed of light, Represents the wavelength of the incident light at the corresponding pixel point of the target image, represents the quantum efficiency function of the camera system, Represents the grayscale value of the pixel at the same position on the three selected target polarization images; Represents a reversible matrix composed of the element values ​​of the Mueller matrix of multiple camera systems, specifically: ; in, Represents grayscale value The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, Represents grayscale value The first row element value of the Mueller matrix of the camera system corresponding to the target polarization image, Represents grayscale value The first row of the Mueller matrix of the camera system corresponding to the target polarization image.

6. The method for image enhancement and target detection of surface defects according to claim 5, It is characterized in that The energy flux density , the energy flux density The calculation process is: ; in, Represents the Stokes vector of the energy flux density of the incident light at the corresponding pixel point of the target image, represents the exposure time of the camera system; ; The energy flux density The energy flux density Stokes vector The first element of .

7. The method for image enhancement and target detection of surface defects according to claim 6, It is characterized in that The initial grayscale value of each pixel on the target image is specifically: ; ; in, Represents the initial grayscale value of the corresponding pixel of the target image, Indicates the exposure time configured on the computer side. Represents the signal-to-noise ratio of the corresponding pixel of the target image, Represents the intensity of the incident light at the corresponding pixel point of the target image, Represents the linear polarization component of the incident light at the corresponding pixel point of the target image.

8. The method for image enhancement and target detection of surface defects according to claim 7, It is characterized in that The final grayscale value of each pixel on the target image is specifically: ; in, Indicates the final grayscale value of the corresponding pixel in the target image.

9. A surface defect image enhancement and target detection system, It is characterized in that The method for image enhancement and target detection of surface defects according to any one of claims 1 to 8 is adopted; The surface defect image enhancement and target detection system comprises: An image acquisition unit, used for simultaneously acquiring at least three target polarization images of a surface defect target having different polarization characteristics; A virtual filtering unit, used for performing virtual polarization filtering according to the target polarization image to obtain an enhanced target image; The surface defect detection model is used to perform defect detection on the enhanced target image.

Citation Information

Patent Citations

  • Laser additive manufacturing workpiece surface defect detection method

    CN115170485A

  • Remote sensing image target tracking method based on multi-scale feature aggregation enhancement

    CN116777953A

  • Metal surface defect detection method based on multi-dimensional optical information fusion and significance guidance

    CN118485618A

  • Polaroid color filtering frame for suppressing reflection interference in infrared touch screen frame

    CN2751317Y

  • Powdered surface monitoring system of additive manufacturing

    US20230302735A1