A method and system for identifying automobile glass screen defects based on machine vision

Through high dynamic range imaging, adaptive histogram equalization and deep learning feature extraction, combined with attention mechanism, the problem of insufficient subjectivity and flexibility of traditional detection methods is solved, and efficient and accurate detection of defect recognition of automotive glass screens is achieved.

CN118247258BActive Publication Date: 2024-11-12深圳市兴悦科技有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410447295.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-15
Publication Date
2024-11-12
Estimated Expiration
2044-04-15

AI Technical Summary

Technical Problem

Traditional automotive glass screen defect detection methods have problems such as insubjectivity and inefficiency in manual inspection, insufficient flexibility in mechanical inspection, and machine vision methods have deteriorated detection performance when image quality is poor or light changes are large.

Method used

Using high dynamic range imaging technology, adaptive histogram equalization processing, deep learning feature extraction and attention mechanism, a defect recognition network is built for automotive glass screen defect recognition, and the detection accuracy is improved through image acquisition, feature extraction and loss function update. efficiency.

Benefits of technology

It significantly improves the accuracy and efficiency of defect recognition, enhances the model's generalization ability to various types and morphological defects, reduces overfitting during training, and improves the performance and reliability of the model in practical applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118247258B_ABST
    Figure CN118247258B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for identifying defects of automobile glass screens based on machine vision, including: S1: using high dynamic range imaging technology to collect images of automobile glass screens to obtain a synthesized automobile glass screen image; S2: adjusting the automobile glass screen image based on adaptive histogram equalization to obtain an adjusted automobile glass screen image; S3: detecting key points of the adjusted automobile glass screen image to generate corresponding intensity centroids, and constructing a feature map of the adjusted automobile glass screen image based on the intensity centroids; S4: constructing a defect recognition network based on the adjusted automobile glass screen image and the feature map to identify defects of the automobile glass screen and obtain a probability distribution of defect categories; S5: using the probability distribution of defect categories and the real defect categories to calculate the loss function, and updating the defect recognition network parameters. The present invention can improve the accuracy and efficiency of defect recognition of automobile glass screens.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of defect recognition, and in particular to a method and system for identifying automobile glass screen defects based on machine vision. Background Art

[0002] With the rapid development of the automobile industry, the quality control of automobile glass screens has become a key link to ensure automobile safety and comfort. Traditional automobile glass screen defect detection mainly relies on manual visual inspection and simple mechanical inspection methods. However, these methods have some shortcomings: manual inspection is affected by factors such as operator experience, fatigue and subjective judgment, and is prone to missed detection and false detection, especially for small defects. The detection effect is not ideal. In addition, manual inspection is inefficient and difficult to meet the needs of mass production. Traditional mechanical inspection methods, such as fixed template-based inspection, can improve the consistency of detection, but lack flexibility and are difficult to adapt to diverse defect types and complex inspection environments. In addition, these methods have limited detection capabilities for non-contact defects such as internal bubbles and microcracks. Although machine vision technology has been applied to automobile glass screen defect detection in recent years, existing machine vision methods still have some limitations. For example, some methods have significantly reduced detection performance when the image quality is poor or the lighting conditions vary greatly. In addition, many methods rely on manually designed feature extraction and selection algorithms, which not only increases the complexity of the system, but also limits the generalization ability and adaptability of the model. Summary of the invention

[0003] In view of this, the present invention provides a method and system for identifying automobile glass screen defects based on machine vision, aiming to introduce advanced technologies such as high dynamic range imaging technology, adaptive histogram equalization processing, deep learning feature extraction and attention mechanism to improve the accuracy and efficiency of defect identification.

[0004] To achieve the above object, the present invention provides a method for identifying automobile glass screen defects based on machine vision, comprising the following steps:

[0005] S1: Use high dynamic range imaging technology to collect images of the automobile glass screen to obtain a synthesized automobile glass screen image;

[0006] S2: adjusting the automobile glass screen image based on adaptive histogram equalization to obtain an adjusted automobile glass screen image;

[0007] S3: Detect key points of the adjusted automobile glass screen image to generate corresponding intensity centroids, and construct a feature map of the adjusted automobile glass screen image based on the intensity centroids;

[0008] S4: Based on the adjusted automobile glass screen image and feature map, a defect recognition network is constructed to identify defects of the automobile glass screen and obtain the probability distribution of defect categories;

[0009] S5: Calculate the loss function using the defect category probability distribution and the true defect category, and update the defect recognition network parameters;

[0010] As a further improvement method of the present invention:

[0011] Optionally, in step S1, using high dynamic range imaging technology to collect images of the automobile glass screen to obtain a synthesized automobile glass screen image includes:

[0012] S11: Image acquisition and fusion:

[0013] Take car glass screen images with different exposure levels and obtain a car glass screen image set {I1,I2,…,I N}, where N represents the number of automobile glass screen images; the automobile glass screen image set is fused based on the weight distribution of the content to obtain the fused automobile glass screen image

[0014]

[0015] Among them, (x, y) represents the coordinates of each point of the car glass screen image; Represents the pixel value of the fused car glass screen image at (x, y); n = 1, 2, ..., N; I n (x, y) represents the pixel value of the nth image in the automobile glass screen image set at (x, y); w(I n (x,y)) is the weight function, according to I n The value of (x,y) is calculated as follows:

[0016]

[0017] in, is the basic weight function, expressed as:

[0018]

[0019] Among them, I n represents the nth image in the automobile glass screen image set; min and max are the minimum and maximum value calculation functions respectively; α(I n (x,y)) represents the adjustment factor, which is calculated as:

[0020]

[0021] Among them, win 3×3 (In (x,y)) represents the n (x, y) is the pixel in the central 3×3 range; Var is the variance calculation function; and I n (x,y) gradient in the x and y directions;

[0022] S12: Adaptive Tone Mapping:

[0023] The adaptive tone mapping function is used on the fused automobile glass screen image to obtain the synthesized automobile glass screen image H:

[0024]

[0025] Wherein, ln represents the natural logarithm;

[0026] Optionally, in step S2, adjusting the automobile glass screen image based on adaptive histogram equalization to obtain the adjusted automobile glass screen image includes:

[0027] S21: Image segmentation and contrast limitation:

[0028] Divide H into non-overlapping blocks, each block is 16×16 in size;

[0029] For each block the histogram is computed and a contrast limit is applied, calculated as:

[0030]

[0031] Among them, small and large functions return the smaller and larger values ​​in the input respectively; i represents the grayscale level of the image; j = 0, 1, ..., Level-1, Level represents the number of grayscale levels of the image; H k (i) represents the number of occurrences of pixels with gray level i in the kth block; H after contrast limitation k (i); th represents the contrast limit threshold;

[0032] S22: Calculate the cumulative distribution function and perform equalization transformation:

[0033] For each contrast-limited block, calculate the cumulative distribution function value:

[0034]

[0035] Among them, C k (i) represents the cumulative distribution function value of gray level i;

[0036] Apply equalization transformation to each pixel of each block, calculate new pixel value according to the pixel value and cumulative distribution function of the corresponding block, and obtain the adjusted car glass screen image

[0037]

[0038] in, Represents the pixel value of the adjusted car glass screen image at (x, y); and C k (0) respectively represent the gray level and the cumulative distribution function value of 0; round represents the rounding function;

[0039] Optionally, in step S3, detecting key points of the adjusted automobile glass screen image to generate corresponding intensity centroids, and constructing a feature map of the adjusted automobile glass screen image based on the intensity centroids includes:

[0040] S31: Key point detection:

[0041] for For each pixel h in the image, select num points on a circle with radius r with h as the center; if there are continuous A point, this The pixel value of points is higher or lower than the pixel value of h. Then h is a key point;

[0042] S32: Key point direction assignment:

[0043] Based on the intensity centroid method, an intensity centroid is assigned to each detected key point, and the intensity centroid δ is calculated as follows:

[0044]

[0045] Among them, circle h It represents the coordinate set of num points selected on the circle with radius r with h as the center; Indicates that the adjusted car glass screen image is The pixel value at ;

[0046] Generate an intensity centroid for each pixel in the adjusted automobile glass screen image to form a feature map Fea of ​​the adjusted automobile glass screen image;

[0047] Optionally, in step S4, based on the adjusted automobile glass screen image and feature map, a defect recognition network is constructed to perform defect recognition on the automobile glass screen, including:

[0048] The defect recognition network consists of a deep feature extraction layer, an attention layer, a feature fusion layer, and a classification layer:

[0049] S41: Deep Feature Extraction:

[0050] Input the adjusted car glass screen image and feature map into the deep feature extraction layer built based on ResNet to obtain the deep feature map:

[0051]

[0052] Among them, F is the depth feature map; θ ResNet are the parameters of ResNet;

[0053] S42: Attention calculation:

[0054] After deep feature extraction, an attention layer is applied to the deep feature map; the attention layer calculates the global average pooling and global maximum pooling of the deep feature map in the channel dimension, and then obtains the channel attention weight through the fully connected layer and activation function:

[0055] W ch =σ(FC(GAP(F))+FC(GMP(F)))

[0056] Among them, σ is the activation function; GAP is the global average pooling function; GMP is the global maximum pooling function; FC is the fully connected layer; W ch is the calculated channel attention weight;

[0057] The spatial attention weight is further calculated based on the channel attention weight:

[0058] W sp =σ(Conv(W ch ·F))

[0059] Among them, Conv represents the convolution operation; W sp is the calculated spatial attention weight;

[0060] S43: Feature fusion and classification:

[0061] The deep feature map weighted by the attention layer is sent to the classification layer for defect classification:

[0062] P = Softmax(FC(W sp ·F))

[0063] Among them, Softmax is the normalized exponential function; P is the probability distribution of defect categories;

[0064] Optionally, in step S5, the defect category probability distribution and the actual defect category are used to calculate the loss function and update the defect recognition network parameters, including:

[0065] S51: Loss function calculation:

[0066] The loss function is calculated using the improved cross entropy loss, which is calculated as follows:

[0067]

[0068] Where D is the total number of defect categories; β d is a predefined weight; P d represents the d-th dimension of P, which indicates the probability that the defect recognition network predicts that the sample belongs to the d-th category; P d The calculation method of P is based on the true defect category. If the true category is d, then P d remains unchanged, otherwise P d Updated to 1-P d ;γ is the regulatory factor;

[0069] S52: Defect identification network parameter update:

[0070] The formula for updating the defect recognition network parameters is:

[0071]

[0072] Among them, η represents the learning rate; λ is the weight decay coefficient; τ is the number of parameter updates; ω τ and ω τ+1 are the defect recognition network parameters after the τth and τ+1th updates respectively; ∈ is a constant to prevent division by zero; and They are respectively the corrected first-order moment estimate and the corrected second-order moment estimate, and are calculated as follows:

[0073]

[0074] u τ =κ1u τ-1 +(1-κ1)G τ

[0075]

[0076] v τ =κ2v τ-1 +(1-κ2)G τ 2

[0077] Among them, κ1 and κ2 represent the decay rates of the first-order moment and the second-order moment respectively; G τrepresents the gradient of the loss function with respect to the defect recognition network parameters during the τth parameter update; u τ and v τ They are the first-order moment estimation and the second-order moment estimation, u0=0, v0=0;

[0078] S53: Defect Identification Web Application:

[0079] For each automobile glass screen image to be defect-identified, the corresponding adjusted automobile glass screen image and feature map are obtained and input into the defect recognition network after parameter update to obtain the defect category probability distribution. The defect with a probability greater than 0.5 is the defect category identified by the defect recognition network;

[0080] The present invention also discloses a vehicle glass screen defect recognition system based on machine vision, comprising:

[0081] Image synthesis module: uses high dynamic range imaging technology to collect images of the car glass screen and obtain a synthesized car glass screen image;

[0082] Equalization module: adjusts the car glass screen image based on adaptive histogram equalization;

[0083] Feature extraction module: constructs the feature map of the adjusted car glass screen image based on the intensity centroid;

[0084] Network construction module: Build a defect recognition network to identify defects on automobile glass screens and obtain the probability distribution of defect categories;

[0085] Network training module: uses the probability distribution of defect categories and the actual defect categories to calculate the loss function and update the defect recognition network parameters;

[0086] Beneficial effects:

[0087] By taking and fusing images at different exposure levels, the present invention can significantly improve the dynamic range of the automotive glass screen image, ensuring that details are preserved in extremely bright or dark areas. This is crucial for subsequent defect identification, especially when the contrast between the defect area and the background is not obvious. By performing adaptive histogram equalization on the image, the present invention further enhances the local contrast of the image, making the defect features more prominent and easier to detect.

[0088] The present invention effectively captures the key features of automotive glass screen defects by detecting key points of the adjusted image and assigning them intensity centroids. By combining the deep feature extraction layer and the attention layer, the present invention is able to automatically extract complex defect features and focus on the most critical information. This not only improves the accuracy of defect identification, but also significantly enhances the model's generalization ability for defects of various types and morphologies.

[0089] By adopting an improved cross entropy loss function and parameter updating method, the present invention achieves more stable and efficient model training when dealing with class imbalance. This method effectively reduces overfitting during training and improves the performance and reliability of the model in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] Figure 1 A schematic diagram of a process of a method for identifying defects in an automobile glass screen based on machine vision according to an embodiment of the present invention; DETAILED DESCRIPTION

[0091] The present invention is further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention belong to the protection scope of the present invention.

[0092] Embodiment 1: A method for identifying defects in automobile glass screens based on machine vision, such as Figure 1 As shown, the following steps are included:

[0093] S1: Use high dynamic range imaging technology to capture images of the car glass screen and obtain a synthetic car glass screen image:

[0094] S11: Image acquisition and fusion:

[0095] Take car glass screen images with different exposure levels and obtain a car glass screen image set {I1,I2,…,I N}, where N represents the number of automobile glass screen images; the automobile glass screen image set is fused based on the weight distribution of the content to obtain the fused automobile glass screen image

[0096]

[0097] Among them, (x, y) represents the coordinates of each point of the car glass screen image; Represents the pixel value of the fused car glass screen image at (x, y); n = 1, 2, ..., N; I n (x, y) represents the pixel value of the nth image in the automobile glass screen image set at (x, y); w(I n (x,y)) is the weight function, according to I n The value of (x,y) is calculated as follows:

[0098]

[0099] in, is the basic weight function, expressed as:

[0100]

[0101] Among them, I n represents the nth image in the automobile glass screen image set; min and max are the minimum and maximum value calculation functions respectively; α(I n (x,y)) represents the adjustment factor, which is calculated as:

[0102]

[0103] Among them, win 3×3 (I n (x,y)) represents the n (x, y) is the pixel in the central 3×3 range; Var is the variance calculation function; and I n (x,y) gradient in the x and y directions;

[0104] S12: Adaptive Tone Mapping:

[0105] The adaptive tone mapping function is used on the fused automobile glass screen image to obtain the synthesized automobile glass screen image H:

[0106]

[0107] Wherein, ln represents the natural logarithm;

[0108] High dynamic range imaging technology can capture a wide range of brightness from extremely bright to extremely dark, which means that more details and contrast can be retained even in environments with extremely complex lighting conditions. For automotive glass screen defect identification, this helps to highlight defect features, especially in reflective or shadowed areas. By synthesizing images with different exposure levels, high dynamic range imaging technology ensures that the features of the defective area in the image are more clearly distinguishable. This is crucial for automatic recognition systems to identify subtle scratches, point defects or other types of damage.

[0109] S2: adjusting the automobile glass screen image based on adaptive histogram equalization to obtain an adjusted automobile glass screen image:

[0110] S21: Image segmentation and contrast limitation:

[0111] Divide H into non-overlapping blocks, each block is 16×16 in size;

[0112] For each block the histogram is computed and a contrast limit is applied, calculated as:

[0113]

[0114] Among them, small and large functions return the smaller and larger values ​​in the input respectively; i represents the grayscale level of the image; j = 0, 1, ..., Level-1, Level represents the number of grayscale levels of the image; H k (i) represents the number of occurrences of pixels with gray level i in the kth block; H after contrast limitation k (i); th represents the contrast limit threshold, which is 5% of the total number of image pixels in this embodiment;

[0115] S22: Calculate the cumulative distribution function and perform equalization transformation:

[0116] For each contrast-limited block, calculate the cumulative distribution function value:

[0117]

[0118] Among them, C k (i) represents the cumulative distribution function value of gray level i;

[0119] Apply equalization transformation to each pixel of each block, calculate new pixel value according to the pixel value and cumulative distribution function of the corresponding block, and obtain the adjusted car glass screen image

[0120]

[0121] in, Represents the pixel value of the adjusted car glass screen image at (x, y); and C k (0) respectively represent the gray level and the cumulative distribution function value of 0; round represents the rounding function;

[0122] Adaptive histogram equalization significantly improves local contrast and makes details more visible by performing histogram equalization on local areas of the image. Since adaptive histogram equalization is applied in local areas of the image, it is very adaptable to changes in illumination of the image as a whole or locally. This is especially important for automotive glass screen images taken under different lighting conditions to ensure consistency and accuracy in defect detection. By improving the local contrast and brightness of the image, adaptive histogram equalization reduces the risk of false detection and missed detection due to image quality issues. This makes the defect detection process more reliable.

[0123] S3: Detect key points of the adjusted automobile glass screen image to generate corresponding intensity centroids, and construct a feature map of the adjusted automobile glass screen image based on the intensity centroids:

[0124] S31: Key point detection:

[0125] for For each pixel h in the image, select num points on a circle with radius r with h as the center; if there are continuous A point, this The pixel value of points is higher or lower than the pixel value of h. In this embodiment is 20% of the pixel value of h, then h is a key point;

[0126] S32: Key point direction assignment:

[0127] Based on the intensity centroid method, an intensity centroid is assigned to each detected key point, and the intensity centroid δ is calculated as follows:

[0128]

[0129] Among them, circle h It represents the coordinate set of num points selected on the circle with radius r with h as the center; Indicates that the adjusted car glass screen image is The pixel value at ;

[0130] Generate an intensity centroid for each pixel in the adjusted automobile glass screen image to form a feature map Fea of ​​the adjusted automobile glass screen image;

[0131] By accurately detecting key points in the image, this method can locate potential defect areas on the car glass screen, such as scratches, cracks, or point defects. This step ensures that the focus of subsequent processing is on the location where defects are most likely to occur, thereby improving the accuracy of recognition. By assigning an intensity centroid to each key point, this method can further refine the key feature information related to the defect. The intensity centroid reflects the pixel intensity distribution of the area around the key point, which helps to highlight the defect features and make them more prominent in the feature map. The feature map constructed based on the intensity centroid provides an enhanced image representation for the defect recognition network. This representation can better capture the details and structural information of the defect area, providing rich and easy-to-learn features for the deep learning model, thereby improving the learning efficiency and recognition performance of the model.

[0132] S4: Based on the adjusted automobile glass screen image and feature map, a defect recognition network is constructed to identify defects on the automobile glass screen and obtain the probability distribution of defect categories:

[0133] The defect recognition network consists of a deep feature extraction layer, an attention layer, a feature fusion layer, and a classification layer:

[0134] S41: Deep Feature Extraction:

[0135] Input the adjusted car glass screen image and feature map into the deep feature extraction layer built based on ResNet to obtain the deep feature map:

[0136]

[0137] Among them, F is the depth feature map; θ ResNet are the parameters of ResNet;

[0138] S42: Attention calculation:

[0139] After deep feature extraction, an attention layer is applied to the deep feature map; the attention layer calculates the global average pooling and global maximum pooling of the deep feature map in the channel dimension, and then obtains the channel attention weight through the fully connected layer and activation function:

[0140] W ch =σ(FC(GAP(F))+FC(GMP(F)))

[0141] Wherein, σ is an activation function, which is a type B function in this embodiment; GAP is a global average pooling function; GMP is a global maximum pooling function; FC is a fully connected layer; W ch is the calculated channel attention weight;

[0142] The spatial attention weight is further calculated based on the channel attention weight:

[0143] W sp =σ(Conv(W ch ·F))

[0144] Among them, Conv represents the convolution operation; W sp is the calculated spatial attention weight;

[0145] S43: Feature fusion and classification:

[0146] The deep feature map weighted by the attention layer is sent to the classification layer for defect classification:

[0147] P = Softmax(FC(W sp ·F))

[0148] Among them, Softmax is the normalized exponential function; P is the probability distribution of defect categories;

[0149] Using ResNet as the underlying deep feature extraction layer, we can deeply explore the complex patterns and details in the adjusted car glass screen image and its feature map. By calculating the channel and spatial attention weights, the model can focus more on the key information related to defect recognition, effectively reducing the interference of background noise and irrelevant information. This focusing mechanism not only improves the recognition accuracy, but also enhances the model's ability to interpret defect features. The feature fusion step considers the adjusted image and the feature map weighted by the attention layer, making more comprehensive use of information from different sources. This method of comprehensively considering the original image and deep learning features makes defect classification more accurate and improves the model's adaptability to complex defect scenarios.

[0150] S5: Calculate the loss function using the defect category probability distribution and the true defect category, and update the defect recognition network parameters:

[0151] S51: Loss function calculation:

[0152] The loss function is calculated using the improved cross entropy loss, which is calculated as follows:

[0153]

[0154] Wherein, D is the total number of defect categories. In this embodiment, D=5, including no defect, scratch defect, bubble defect, crack defect, and atomization defect; β d are predefined weights. In this embodiment, β0=0.1, β2=β3=β4=β5=0.3; P d represents the d-th dimension of P, which indicates the probability that the defect recognition network predicts that the sample belongs to the d-th category; P d The calculation method of P is based on the true defect category. If the true category is d, then P d remains unchanged, otherwise P d Updated to 1-P d ; γ is the adjustment factor, which is 0.2 in this embodiment;

[0155] S52: Defect identification network parameter update:

[0156] The formula for updating the defect recognition network parameters is:

[0157]

[0158] Wherein, η represents the learning rate, which is 0.002 in this embodiment; λ is the weight attenuation coefficient, which is 0.7 in this embodiment; τ is the number of parameter updates; ω τ and ω τ+1 are the defect recognition network parameters after the τth and τ+1th updates respectively; ∈ is a constant to prevent division by zero, ∈=0.0000001; and They are respectively the corrected first-order moment estimate and the corrected second-order moment estimate, and are calculated as follows:

[0159]

[0160] u τ =κ1u τ-1 +(1-κ1)G τ

[0161]

[0162] v τ =κ2v τ-1 +(1-κ2)G τ 2

[0163] Wherein, κ1 and κ2 represent the attenuation rates of the first-order moment and the second-order moment, respectively, which are 0.9 and 0.98 in this embodiment; G τ represents the gradient of the loss function with respect to the defect recognition network parameters during the τth parameter update; u τ and v τ They are the first-order moment estimation and the second-order moment estimation, u0=0, v0=0;

[0164] S53: Defect Identification Web Application:

[0165] For each automobile glass screen image to be defect-identified, the corresponding adjusted automobile glass screen image and feature map are obtained and input into the defect recognition network after parameter update to obtain the defect category probability distribution. The defect with a probability greater than 0.5 is the defect category identified by the defect recognition network;

[0166] By introducing an improved cross entropy loss, especially by setting different weights and adjustment factors for different defect categories, this method can pay more attention to rare or difficult-to-identify defect types. The introduction of the adjustment factor allows the model to pay less attention to samples that are already easily correctly classified during training, and focus more on samples that are difficult to classify. This method of dynamically adjusting the learning focus helps to accelerate the learning progress of the model and improve the generalization ability of the final model. Using the modified first-order moment estimate and second-order moment estimate for parameter update can provide a robust parameter update mechanism for the defect recognition network. This mechanism helps to avoid the problem of gradient explosion or disappearance during training, ensuring the stability and efficiency of training.

[0167] Embodiment 2: The present invention also discloses a vehicle glass screen defect recognition system based on machine vision, comprising the following five modules:

[0168] Image synthesis module: uses high dynamic range imaging technology to collect images of the car glass screen and obtain a synthesized car glass screen image;

[0169] Equalization module: adjusts the car glass screen image based on adaptive histogram equalization;

[0170] Feature extraction module: constructs the feature map of the adjusted car glass screen image based on the intensity centroid;

[0171] Network construction module: Build a defect recognition network to identify defects on automobile glass screens and obtain the probability distribution of defect categories;

[0172] Network training module: uses the probability distribution of defect categories and the actual defect categories to calculate the loss function and update the defect recognition network parameters;

[0173] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0174] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0175] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for identifying automobile glass screen defects based on machine vision, characterized in that: The following steps are involved: S1: Use high dynamic range imaging technology to collect images of the automobile glass screen to obtain a synthesized automobile glass screen image; The step S1 includes the following steps: S11: Image acquisition and fusion: Take car glass screen images at different exposure levels to obtain a set of car glass screen images ,in Represents the number of automobile glass screen images; fuses the automobile glass screen image set based on content weight distribution to obtain the fused automobile glass screen image : ; in, Represents the coordinates of each point of the car glass screen image; Indicates that the fused car glass screen image is The pixel value at ; ; Represents the car glass screen image collection The image is The pixel value at ; is the weight function, according to The value of is calculated as follows: ; in, is the basic weight function, expressed as: ; in, Represents the car glass screen image collection images; and Calculate functions for minimum and maximum values ​​separately; represents the adjustment factor, which is calculated as: ; in, Indicates Centered Pixels within range; is the variance calculation function; and Respectively exist and Directional gradient; S12: Adaptive Tone Mapping: Use an adaptive tone mapping function on the fused car glass screen image to obtain a synthesized car glass screen image : ; in, represents the natural logarithm; S2: Adjust the automobile glass screen image based on adaptive histogram equalization to obtain an adjusted automobile glass screen image ; S3: Detect key points of the adjusted automobile glass screen image to generate corresponding intensity centroids, and construct a feature map of the adjusted automobile glass screen image based on the intensity centroids; The step S3 includes the following steps: S31: Key point detection: for Each pixel in ,by Centered at radius Select on the circle point; if this There are continuous A point, this The pixel value ratio of The pixel value is above or below the threshold ,but It is a key point; S32: Key point direction assignment: Based on the intensity centroid method, an intensity centroid is assigned to each detected key point. The calculation method is: ; in, Indicates Centered at radius Select on the circle A set of coordinates of points; ; Indicates that the adjusted car glass screen image is The pixel value at ; Generate the intensity centroid for each pixel in the adjusted automobile glass screen image to form the feature map of the adjusted automobile glass screen image ; S4: Based on the adjusted automobile glass screen image and feature map, a defect recognition network is constructed to identify defects on the automobile glass screen and obtain the probability distribution of defect categories. ; S5: Calculate the loss function using the defect category probability distribution and the true defect category, and update the defect recognition network parameters; The step S5 includes the following steps: S51: Loss function calculation: The loss function is calculated using the improved cross entropy loss, which is calculated as follows: ; in, is the total number of defect categories; are predefined weights; express No. dimension, indicating that the defect recognition network predicts that the sample belongs to The probability of each category; The calculation method is based on the true defect category. If the true category is ,So Remain unchanged, otherwise Updated to ; is the regulating factor; S52: Defect identification network parameter update: The formula for updating the defect recognition network parameters is: ; in, represents the learning rate; is the weight attenuation coefficient; is the number of parameter updates; and Respectively Second and The updated defect recognition network parameters; is an integer that prevents division by zero; and They are respectively the corrected first-order moment estimate and the corrected second-order moment estimate, and are calculated as follows: ; ; ; ; in, and denote the decay rates of the first-order moment and the second-order moment respectively; Indicates The gradient of the loss function with respect to the defect recognition network parameters during the parameter update; and are the first-order moment estimate and the second-order moment estimate, respectively. , ; S53: Defect Identification Web Application: For each automobile glass screen image to be identified for defects, the corresponding adjusted automobile glass screen image and feature map are obtained and input into the defect recognition network after parameter update to obtain the defect category probability distribution. Defects with a probability greater than 0.5 are the defect categories identified by the defect recognition network.

2. The method for identifying automobile glass screen defects based on machine vision according to claim 1, characterized in that: The step S2 includes the following steps: S21: Image Blocking and Contrast Limitation: Will Divide into non-overlapping blocks, each block is of size ; For each block the histogram is computed and a contrast limit is applied, calculated as: ; in, and The function returns the smaller and larger value of the input respectively; Represents the grayscale of the image; , Indicates the number of gray levels of the image; Indicates blocks, gray level is The number of times the pixel appears; Indicates the contrast-limited ; represents the contrast limit threshold; S22: Calculate the cumulative distribution function and perform equalization transformation: For each contrast-limited block, calculate the cumulative distribution function value: ; in, Indicates the gray level is The cumulative distribution function value of ; Apply equalization transformation to each pixel of each block, calculate new pixel value according to the pixel value and cumulative distribution function of the corresponding block, and obtain the adjusted car glass screen image : ; in, Indicates that the adjusted car glass screen image is The pixel value at ; and Respectively represent the gray level and the cumulative distribution function value of 0; Represents the rounding function.

3. The method for identifying automobile glass screen defects based on machine vision according to claim 2, characterized in that: The step S4 includes the following steps: The defect recognition network consists of a deep feature extraction layer, an attention layer, a feature fusion layer, and a classification layer: S41: Deep Feature Extraction: Input the adjusted car glass screen image and feature map into the deep feature extraction layer built based on ResNet to obtain the deep feature map: ; in, is the deep feature map; are the parameters of ResNet; S42: Attention calculation: After deep feature extraction, an attention layer is applied to the deep feature map; the attention layer calculates the global average pooling and global maximum pooling of the deep feature map in the channel dimension, and then obtains the channel attention weight through the fully connected layer and activation function: ; in, is the activation function; is the global average pooling function; is the global maximum pooling function; is a fully connected layer; is the calculated channel attention weight; The spatial attention weight is further calculated based on the channel attention weight: ; in, Represents the convolution operation; is the calculated spatial attention weight; S43: Feature fusion and classification: The deep feature map weighted by the attention layer is sent to the classification layer for defect classification: ; in, is the normalized exponential function; is the defect class probability distribution.

4. A machine vision-based automobile glass screen defect recognition system, characterized in that: include: Image synthesis module: uses high dynamic range imaging technology to collect images of the car glass screen and obtain a synthesized car glass screen image; Equalization module: adjusts the car glass screen image based on adaptive histogram equalization; Feature extraction module: constructs the feature map of the adjusted car glass screen image based on the intensity centroid; Network construction module: Build a defect recognition network to identify defects on automobile glass screens and obtain the probability distribution of defect categories; Network training module: uses the probability distribution of defect categories and the actual defect categories to calculate the loss function and update the defect recognition network parameters; To realize the automobile glass screen defect recognition method based on machine vision as described in any one of claims 1-3.