Electronic component surface defect detection method and system based on artificial intelligence

Abstract

The embodiment of the invention relates to the technical field of defect detection, and particularly discloses an electronic component surface defect detection method and system based on artificial intelligence. According to the embodiment of the invention, a plurality of electronic component samples are shot, and sample marking information is obtained; constructing a defect detection model; verification is carried out, and an optimization detection model is generated; acquiring a detection shooting image in real time; and importing into the optimization detection model, and generating and displaying a defect detection result. The method can process a plurality of electronic component samples, train and construct a defect detection model, verify and optimize the defect detection model in a model verification period, generate an optimized detection model, obtain a detection shot image in real time, perform real-time defect detection, and generate and display a defect detection result. Therefore, real-time surface defect detection can be carried out in the production and processing process of the electronic component, an independent defect detection process does not need to be developed, the production and processing time of the electronic component is effectively shortened, and the production cost is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of defect detection, and in particular to an electronic component surface defect detection method and system based on artificial intelligence. Background Art

[0002] Surface defect detection of electronic components is to ensure that electronic components meet the predetermined quality standards during the manufacturing process, and to identify and evaluate various defects that may exist on their surfaces through specific detection means and methods. These defects may include but are not limited to scratches, dents, abrasions, rust, mold, bubbles, pinholes, pitting, surface cracks, delamination, wrinkles, etc. They not only affect the appearance quality of the product, but may also directly affect the performance, life and reliability of the product.

[0003] In the prior art, defect detection on the surface of electronic components is usually carried out by manual detection or semi-automatic detection. The defect detection process is complicated and time-consuming. It is not convenient to perform real-time surface defect detection during the production and processing of electronic components. Instead, it is necessary to open up a separate defect detection process outside the production and processing of electronic components, which prolongs the production and processing time of electronic components and invisibly increases production costs. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide an electronic component surface defect detection method and system based on artificial intelligence, aiming to solve the problems raised in the background technology.

[0005] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions: An electronic component surface defect detection method based on artificial intelligence, the method specifically comprises the following steps: Photographing a plurality of electronic component samples to obtain a plurality of sample photographed images, processing the plurality of sample photographed images to generate a plurality of sample processed images, and obtaining sample marking information; Based on the sample marking information, defect feature analysis is performed on the plurality of sample processed images, and based on a preset training model, a defect detection model is trained and constructed; In a preset model verification period, a plurality of verification processing images and verification mark information are obtained, the defect detection model is verified and optimized, and an optimized detection model is generated; Based on artificial intelligence technology, the detection image is acquired in real time, and the detection image is processed to generate a detection processing image; The inspection processed image is imported into the optimized inspection model, real-time defect detection is performed on the inspection processed image, and defect detection results are generated and displayed.

[0006] As a further limitation of the technical solution of the embodiment of the present invention, the step of photographing a plurality of electronic component samples to obtain a plurality of sample photographed images, processing the plurality of sample photographed images to generate a plurality of sample processed images, and obtaining sample marking information specifically includes the following steps: Get device shooting parameters; According to the device shooting parameters, a plurality of electronic component samples are photographed to obtain a plurality of sample photographed images; Performing image standardization, image denoising and image enhancement processing on the plurality of sample captured images to generate a plurality of sample processed images; Get sample label information.

[0007] As a further limitation of the technical solution of the embodiment of the present invention, the defect feature analysis is performed on the plurality of sample processing images based on the sample label information, and the training and construction of the defect detection model based on the preset training model specifically includes the following steps: Based on the sample marking information, performing defect analysis on the plurality of sample processing images to determine sample defect types and sample defect locations of the plurality of sample processing images; Generating sample defect information of a plurality of sample processed images according to a plurality of sample defect types and a plurality of sample defect positions; Constructing a training set according to the plurality of sample processed images and the plurality of corresponding sample defect information; Based on the preset training model, the training set is imported into the model training to construct a defect detection model.

[0008] As a further limitation of the technical solution of the embodiment of the present invention, the method of obtaining a plurality of verification processing images and verification mark information during a preset model verification period, verifying and optimizing the defect detection model, and generating an optimized detection model specifically includes the following steps: Create batch verification tasks during the preset model verification period; According to the batch verification task, a plurality of verification processing images and verification mark information are obtained; Importing the plurality of verification processed images into the defect detection model, and obtaining a plurality of verification detection results derived from the defect detection model; According to the verification mark information, verify and compare the multiple verification test results to screen multiple verification error results; According to the plurality of verification failure results, screening a plurality of verification failure images and corresponding marking information; The defect detection model is optimized and trained according to the plurality of verification failure images and the corresponding marking information to generate an optimized detection model.

[0009] As a further limitation of the technical solution of the embodiment of the present invention, the method of acquiring the detection image in real time based on artificial intelligence technology, and processing the detection image to generate the detection processed image specifically includes the following steps: Create real-time detection tasks based on artificial intelligence technology; According to the real-time detection task, acquiring detection and shooting images in real time; The detection shot image is subjected to image standardization, image denoising and image enhancement processing to generate a detection processed image.

[0010] As a further limitation of the technical solution of the embodiment of the present invention, the step of importing the detected processed image into the optimized detection model, performing real-time defect detection on the detected processed image, and generating and displaying the defect detection result specifically includes the following steps: Importing the detected processed image into the optimized detection model; Obtaining defect detection results derived from detection using the optimized detection model; Get the result display address; The defect detection result is transmitted and displayed according to the result display address.

[0011] An electronic component surface defect detection system based on artificial intelligence, the system comprises a sample shooting processing unit, a detection model training unit, a detection model optimization unit, a detection shooting processing unit and a real-time defect detection unit, wherein: A sample shooting and processing unit, used to shoot a plurality of electronic component samples, obtain a plurality of sample shooting images, process the plurality of sample shooting images, generate a plurality of sample processing images, and obtain sample marking information; A detection model training unit, used to perform defect feature analysis on a plurality of sample processed images based on the sample label information, and train and construct a defect detection model based on a preset training model; A detection model optimization unit, used to obtain a plurality of verification processing images and verification mark information during a preset model verification period, verify and optimize the defect detection model, and generate an optimized detection model; A detection shooting processing unit, used to acquire detection shooting images in real time based on artificial intelligence technology, and process the detection shooting images to generate detection processing images; A real-time defect detection unit is used to import the detection processing image into the optimized detection model, perform real-time defect detection on the detection processing image, and generate and display defect detection results.

[0012] As a further limitation of the technical solution of the embodiment of the present invention, the sample shooting processing unit specifically includes: A parameter acquisition module, used to obtain device shooting parameters; A sample shooting module, used to shoot a plurality of electronic component samples according to the device shooting parameters to obtain a plurality of sample shooting images; An image processing module, used for performing image standardization, image denoising and image enhancement processing on the plurality of sample captured images to generate a plurality of sample processed images; The information acquisition module is used to obtain sample labeling information.

[0013] As a further limitation of the technical solution of the embodiment of the present invention, the detection model training unit specifically includes: A defect analysis module, configured to perform defect analysis on the plurality of sample processing images based on the sample marking information, and determine sample defect types and sample defect locations of the plurality of sample processing images; An information generating module, used for generating sample defect information of a plurality of sample processed images according to a plurality of sample defect types and a plurality of sample defect positions; A training set construction module, used to construct a training set according to a plurality of sample processed images and a plurality of corresponding sample defect information; The model building module is used to import the model training to the training set based on the preset training model to build a defect detection model.

[0014] As a further limitation of the technical solution of the embodiment of the present invention, the detection model optimization unit specifically includes: The task creation module is used to create batch verification tasks during the preset model verification period; A task processing module, used for obtaining a plurality of verification processing images and verification mark information according to the batch verification task; A verification import module, used to import the multiple verification processed images into the defect detection model, and obtain multiple verification detection results exported by the defect detection model; A verification comparison module, used to verify and compare the multiple verification test results according to the verification mark information, and screen multiple verification error results; An error screening module, used for screening a plurality of verification error images and corresponding marking information according to a plurality of verification error results; The model optimization module is used to optimize the defect detection model according to the multiple verification failure images and the corresponding marking information to generate an optimized detection model.

[0015] Compared with the prior art, the present invention has the following beneficial effects: The embodiment of the present invention photographs multiple electronic component samples and obtains sample marking information; constructs a defect detection model; performs verification to generate an optimized detection model; obtains the detection and photographed images in real time; imports the optimized detection model, and generates and displays defect detection results. It is possible to process multiple electronic component samples, train and construct defect detection models, and verify and optimize the defect detection models during the model verification period to generate optimized detection models, and then obtain the detection and photographed images in real time, perform real-time defect detection, and generate and display defect detection results, so that real-time surface defect detection can be performed during the production and processing of electronic components without opening up a separate defect detection process, effectively shortening the production and processing time of electronic components and reducing production costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0017] Figure 1 A flow chart of a method provided by an embodiment of the present invention is shown.

[0018] Figure 2 A flow chart of electronic component sample processing in the method provided by an embodiment of the present invention is shown.

[0019] Figure 3 A flow chart of training and constructing a defect detection model in the method provided in an embodiment of the present invention is shown.

[0020] Figure 4 A flow chart of generating an optimized detection model in the method provided in an embodiment of the present invention is shown.

[0021] Figure 5 A flow chart of generating a detection processing image in the method provided by an embodiment of the present invention is shown.

[0022] Figure 6 A flow chart of generating defect detection results in the method provided in an embodiment of the present invention is shown.

[0023] Figure 7 The application architecture diagram of the system provided by the embodiment of the present invention is shown.

[0024] Figure 8 The structure block diagram of the sample shooting processing unit in the system provided by the embodiment of the present invention is shown.

[0025] Fig. 9 A structural block diagram of a detection model training unit in a system provided by an embodiment of the present invention is shown.

[0026] Fig.10 A structural block diagram of a detection model optimization unit in a system provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0028] It is understandable that in the prior art, defect detection on the surface of electronic components is usually carried out by manual detection or semi-automatic detection. The defect detection process is complicated and time-consuming, which makes it inconvenient to perform real-time surface defect detection during the production and processing of electronic components. Instead, it is necessary to open up a separate defect detection process outside the production and processing of electronic components, which prolongs the production and processing time of electronic components and invisibly increases production costs.

[0029] To solve the above problems, the embodiment of the present invention shoots multiple electronic component samples, obtains multiple sample shooting images, processes multiple sample shooting images, generates multiple sample processing images, and obtains sample marking information; based on the sample marking information, the defect feature analysis is performed on the multiple sample processing images, and based on the preset training model, the defect detection model is trained and constructed; in the preset model verification period, multiple verification processing images and verification marking information are obtained, the defect detection model is verified and optimized, and an optimized detection model is generated; based on artificial intelligence technology, the detection shooting images are obtained in real time, and the detection shooting images are processed to generate detection processing images; the detection processing images are imported into the optimized detection model, and the detection processing images are detected in real time, and the defect detection results are generated and displayed. It is possible to process multiple electronic component samples, train and construct defect detection models, and verify and optimize the defect detection models during the model verification period to generate optimized detection models, and then obtain the detection shooting images in real time, perform real-time defect detection, generate and display defect detection results, so that real-time surface defect detection can be performed in the production and processing process of electronic components without opening up a separate defect detection process, effectively shortening the production and processing time of electronic components and reducing production costs.

[0030] Figure 1 A flow chart of a method provided by an embodiment of the present invention is shown.

[0031] Specifically, the method for detecting surface defects of electronic components based on artificial intelligence comprises the following steps: Step S101 , photographing a plurality of electronic component samples to obtain a plurality of sample photographed images, processing the plurality of sample photographed images to generate a plurality of sample processed images, and obtaining sample marking information.

[0032] In an embodiment of the present invention, by obtaining device shooting parameters, multiple electronic component samples are photographed according to the device shooting parameters, and sample shot images of the multiple electronic component samples are obtained. Then, the multiple sample shot images are subjected to image standardization processing, the brightness and contrast are adjusted, and grayscale or color space conversion is performed. Then, image denoising processing is performed, and median filtering, bilateral filtering and other methods are used to reduce image noise. Image enhancement processing is performed, and data diversity is increased through operations such as rotation, flipping, scaling, and shearing to prevent overfitting. Multiple sample processed images are generated, and sample marking information of manual defect marking of multiple electronic component samples is obtained.

[0033] Specifically, Figure 2 A flow chart of electronic component sample processing in the method provided by an embodiment of the present invention is shown.

[0034] Among them, in the preferred embodiment provided by the present invention, the plurality of electronic component samples are photographed to obtain a plurality of sample photographed images, and the plurality of sample photographed images are processed to generate a plurality of sample processed images, and the acquisition of sample marking information specifically includes the following steps: Step S1011, obtaining device shooting parameters; Step S1012, photographing a plurality of electronic component samples according to the device photographing parameters to obtain a plurality of sample photographing images; Step S1013, performing image standardization, image denoising and image enhancement processing on the plurality of sample captured images to generate a plurality of sample processed images; Step S1014, obtaining sample labeling information.

[0035] Specifically, performing image standardization, image denoising and image enhancement processing on the plurality of sample captured images to generate a plurality of sample processed images, the specific steps are as follows: Extract sample pixels, brightness values ​​of pixels and original pixel values ​​from sample captured images; Based on the brightness value of each pixel, the brightness mean of the sample captured image is calculated; Based on the brightness value of each pixel, the standard deviation of the brightness value is calculated; Subtract the brightness mean of the sample image from the brightness value of each pixel, and then divide it by the standard deviation of the brightness value to obtain the standardized brightness value; Assign the standardized brightness value to the sample pixel to generate a standardized image; Determine the denoising weight through the Gaussian kernel function; Extracting standardized pixels from the standardized image; For each standardized pixel, the weighted average of the pixels around the current standardized pixel is calculated using the denoising weight, and then the weighted average of the pixels around the current standardized pixel is assigned to the standardized pixel to generate a denoised image; Extract denoised pixels from the denoised image; Based on the denoised pixel, the Laplace value of the pixel is obtained by calculation, and the Laplace value is added to the original pixel value to obtain the enhanced value of the pixel; Assign the enhanced value of the pixel to the denoised pixel to generate an enhanced image; Determining sample processing weights based on the standardized image, the denoised image, and the enhanced image; A sample processing image is generated according to the standardized brightness value, the weighted average of the pixels around the current standardized pixel, and the Laplace value of the pixel, combined with the sample processing weight.

[0036] This step makes the brightness distribution of the image more uniform through standardization, providing high-quality basic data for subsequent denoising, enhancement and labeling operations. This not only improves the stability and accuracy of image processing, but also enhances the flexibility and efficiency of image processing.

[0037] Furthermore, the electronic component surface defect detection method based on artificial intelligence also includes the following steps: Step S102: Based on the sample labeling information, defect feature analysis is performed on the plurality of sample processed images, and a defect detection model is trained and constructed based on a preset training model.

[0038] In an embodiment of the present invention, defect analysis is performed on multiple sample processing images based on sample marking information to determine sample defect types and sample defect positions of the multiple sample processing images, and then sample defect information of the multiple sample processing images is generated according to the multiple sample defect types and the multiple sample defect positions. A training set is constructed by associating and arranging the multiple sample processing images and the multiple sample defect information. Then, based on a preset training model, the training set is imported into the training model for model training to construct a defect detection model.

[0039] Specifically, Figure 3 A flow chart of training and constructing a defect detection model in the method provided in an embodiment of the present invention is shown.

[0040] Among them, in the preferred embodiment provided by the present invention, the defect feature analysis is performed on the plurality of sample processing images based on the sample marking information, and the training and construction of the defect detection model based on the preset training model specifically includes the following steps: Step S1021, based on the sample marking information, performing defect analysis on the plurality of sample processed images to determine sample defect types and sample defect locations of the plurality of sample processed images; Step S1022, generating sample defect information of a plurality of sample processed images according to a plurality of sample defect types and a plurality of sample defect positions; Step S1023, constructing a training set according to the plurality of sample processed images and the plurality of corresponding sample defect information; Step S1024: Based on a preset training model, the training set is imported for model training to construct a defect detection model.

[0041] Furthermore, the electronic component surface defect detection method based on artificial intelligence also includes the following steps: Step S103, in a preset model verification period, a plurality of verification processing images and verification mark information are obtained, the defect detection model is verified and optimized, and an optimized detection model is generated.

[0042] In an embodiment of the present invention, a batch verification task is created during a preset model verification period, and based on the batch verification task, a plurality of verification processing images and verification mark information are obtained, and by importing the plurality of verification processing images into the defect detection model, a plurality of verification detection results exported by the defect detection model are obtained, and then based on the verification mark information, the plurality of verification detection results are verified and compared, a plurality of verification error results are screened, and according to the plurality of verification error results, a plurality of verification error images and corresponding mark information are screened, and based on the plurality of verification error results, the defect detection model is optimized and trained to generate an optimized detection model.

[0043] Specifically, Figure 4 A flow chart of generating an optimized detection model in the method provided in an embodiment of the present invention is shown.

[0044] Among them, in the preferred embodiment provided by the present invention, the method of obtaining a plurality of verification processing images and verification mark information during a preset model verification period, verifying and optimizing the defect detection model, and generating an optimized detection model specifically includes the following steps: Step S1031, creating a batch verification task in a preset model verification period; Step S1032, obtaining a plurality of verification processing images and verification mark information according to the batch verification task; Step S1033, importing the plurality of verification processed images into the defect detection model, and obtaining a plurality of verification detection results derived from the defect detection model; Step S1034, verifying and comparing the plurality of verification test results according to the verification mark information, and screening a plurality of verification error results; Step S1035, screening a plurality of verification failure images and corresponding marking information according to the plurality of verification failure results; Step S1036, optimizing and training the defect detection model based on the plurality of verification failure images and the corresponding marking information to generate an optimized detection model.

[0045] Further, based on the verification mark information, multiple verification test results are verified and compared, and multiple verification error results are screened. The specific steps are as follows: Acquire real defect information of the sample, wherein the real defect information of the sample includes the sample defect type and the sample defect position; Associating each verification processing image, verification mark information, and verification test result to obtain a set of paired verification data; wherein each set of verification data includes a verification processing image, defect information, and verification test result; Determine whether the sample defect type predicted by the model is consistent with the sample defect type in the actual defect information of the sample. If not, record the classification error value accordingly; Calculate and compare the intersection-and-union ratio of the sample defect position predicted by the model and the sample defect position in the actual defect information of the sample. If the intersection-and-union ratio is lower than the preset standard value, record the positioning error value; Based on the classification error value and the positioning error value, a weighted sum is performed in combination with a preset comprehensive difference weight to obtain a comprehensive difference value; Preset dynamic thresholds based on different sample defect types; The comprehensive difference value is compared with the dynamic threshold. If the comprehensive difference value exceeds the dynamic threshold, the target sample is judged as a verification error result, and a verification error result set is generated; wherein the error result set includes all samples that exceed the dynamic threshold and the verification processing images and verification mark information corresponding to the samples.

[0046] This step improves the comprehensiveness of verification through comprehensive difference values ​​and enhances the flexibility of screening based on dynamic threshold adjustment; at the same time, it reduces computing resource consumption by accurately screening error samples to focus on key problem samples.

[0047] Furthermore, the electronic component surface defect detection method based on artificial intelligence also includes the following steps: Step S104, based on artificial intelligence technology, the detection shot image is acquired in real time, and the detection shot image is processed to generate a detection processed image.

[0048] In an embodiment of the present invention, the optimized detection model is directly arranged in the production line system. During the production and processing of electronic components, a real-time detection task is created based on artificial intelligence technology. According to the real-time detection task, the detection image is acquired in real time. By performing image standardization, image denoising and image enhancement processing on the detection image, a detection processing image is generated.

[0049] Specifically, Figure 5 A flow chart of generating a detection processing image in the method provided by an embodiment of the present invention is shown.

[0050] Among them, in the preferred embodiment provided by the present invention, the real-time acquisition of the detection shot image based on artificial intelligence technology, and the processing of the detection shot image to generate the detection processed image specifically include the following steps: Step S1041, creating a real-time detection task based on artificial intelligence technology; Step S1042, acquiring detection images in real time according to the real-time detection task; Step S1043, performing image standardization, image denoising and image enhancement processing on the detection captured image to generate a detection processed image.

[0051] Specifically, the detection image is subjected to image standardization, image denoising and image enhancement processing to generate a detection processed image. The specific steps are as follows: Obtaining detection shooting images and environmental parameters, wherein the environmental parameters include information of the shooting device; Based on the detection shot image, calculate the pixel mean value of the detection shot image and the standard deviation of the detection shot image; Dynamically adjust the normalization parameters based on the pixel mean of the detected image, the standard deviation of the detected image, and the environmental parameters; Applying the standardized parameters to the detection image to generate a standardized detection image; Performing wavelet decomposition on the standardized detection shot image to obtain a detection shot image after wavelet decomposition; In the detection shooting image after wavelet decomposition, the non-local similarity block matching method is used to smooth the noise in the area where the texture details are retained to obtain the denoised detection shooting image; Build an attention map generation module containing a convolutional neural network; The denoised inspection image is input into the attention map generation module to obtain the attention map, and the key areas that need to be enhanced are marked in the attention map based on the sample defect location; Based on the attention weight, the pixel values ​​in the key area are nonlinearly stretched, and the brightness of the non-key area is suppressed to generate an enhanced detection image. The enhanced detection captured image is adjusted to the output size and color space required by the optimized detection model, and finally the detection processed image is output.

[0052] This step solves the problem of image quality fluctuation caused by differences in shooting equipment by dynamically adjusting brightness scaling and contrast offset parameters.

[0053] Furthermore, the electronic component surface defect detection method based on artificial intelligence also includes the following steps: Step S105, importing the inspection processed image into the optimized inspection model, performing real-time defect detection on the inspection processed image, and generating and displaying defect detection results.

[0054] In an embodiment of the present invention, the inspection and processing image is imported into the optimized inspection model, and then the defect detection result derived from the inspection by the optimized inspection model is obtained, and the result display address is obtained, and then the defect detection result is transmitted and displayed according to the result display address.

[0055] Specifically, Figure 6 A flow chart of generating defect detection results in the method provided in an embodiment of the present invention is shown.

[0056] Among them, in the preferred embodiment provided by the present invention, the step of importing the detected processed image into the optimized detection model, performing real-time defect detection on the detected processed image, and generating and displaying the defect detection result specifically includes the following steps: Step S1051, importing the detected processed image into the optimized detection model; Step S1052, obtaining the defect detection result derived from the detection by the optimized detection model; Step S1053, obtaining the result display address; Step S1054: transmit and display the defect detection result according to the result display address.

[0057] Specifically, the defect detection results derived from the optimized detection model are obtained, and the specific steps are as follows: Input the inspection processed image into the optimized inspection model, analyze the potential defect area in the inspection processed image to generate an original monitoring result set; wherein the original monitoring result set includes a data list of detection frame coordinates, confidence scores, and defect category labels; Generate a dynamic category weight list based on sample defect types; The original monitoring result set is screened according to the sample defect type to generate a target defect subset; Based on historical data, dynamically modified weight values ​​are extracted from the dynamic category weight list; Based on the dynamic correction weight value and the confidence score in the original monitoring result set, a corrected confidence score is obtained through weighted calculation, and the corrected confidence score is returned to the original monitoring result set to generate a corrected detection result set; wherein the corrected detection result set includes a data list of detection box coordinates, corrected confidence scores, and defect category labels; Setting a baseline threshold based on the training data used during the training of the defect detection model; The average value of sample failure rate is calculated through historical data; The dynamic threshold is limited according to the average value of the reference threshold and the unqualified rate to obtain the dynamic judgment threshold; extracting revised confidence scores from the revised set of detection results, and comparing each revised confidence score with a dynamic threshold; Performing confidence attenuation processing on the corrected confidence scores that do not reach the dynamic threshold to obtain attenuated confidence scores; The attenuated confidence scores are filtered to generate and export defect detection results; wherein the defect detection results include a data list of detection frame coordinates, attenuated confidence scores, and defect category labels.

[0058] This step solves the problem that fixed weights cannot adapt to different production batches by dynamically determining the threshold, thereby improving the flexibility and accuracy of detection; the dynamic threshold is linked to historical production data to provide feedback to optimize detection sensitivity and reduce missed detections.

[0059] Furthermore, Figure 7 The application architecture diagram of the system provided by the embodiment of the present invention is shown.

[0060] Among them, in another preferred embodiment provided by the present invention, the electronic component surface defect detection system based on artificial intelligence includes: The sample shooting and processing unit 101 is used to shoot a plurality of electronic component samples to obtain a plurality of sample shooting images, process the plurality of sample shooting images to generate a plurality of sample processing images, and obtain sample marking information.

[0061] In an embodiment of the present invention, the sample shooting processing unit 101 obtains device shooting parameters, shoots multiple electronic component samples according to the device shooting parameters, obtains sample shooting images of the multiple electronic component samples, and then performs image standardization on the multiple sample shooting images, adjusts brightness and contrast, performs grayscale or color space conversion, and then performs image denoising, uses median filtering, bilateral filtering and other methods to reduce image noise, and performs image enhancement processing, increases data diversity through operations such as rotation, flipping, scaling, and shearing to prevent overfitting, generates multiple sample processing images, and obtains sample marking information for manually marking defects on multiple electronic component samples.

[0062] Specifically, Figure 8 It shows a structural block diagram of the sample shooting processing unit 101 in the system provided by an embodiment of the present invention.

[0063] In a preferred embodiment of the present invention, the sample capture processing unit 101 specifically includes: The parameter acquisition module 1011 is used to acquire the device shooting parameters; A sample shooting module 1012 is used to shoot a plurality of electronic component samples according to the device shooting parameters to obtain a plurality of sample shooting images; An image processing module 1013 is used to perform image standardization, image denoising and image enhancement processing on the plurality of sample captured images to generate a plurality of sample processed images; The information acquisition module 1014 is used to acquire sample label information.

[0064] Furthermore, the electronic component surface defect detection system based on artificial intelligence also includes: The detection model training unit 102 is used to perform defect feature analysis on the plurality of sample processed images based on the sample labeling information, and to train and construct a defect detection model based on a preset training model.

[0065] In an embodiment of the present invention, the detection model training unit 102 performs defect analysis on multiple sample processed images based on sample labeling information, determines sample defect types and sample defect positions of the multiple sample processed images, and then generates sample defect information of the multiple sample processed images according to the multiple sample defect types and the multiple sample defect positions, constructs a training set by associating and arranging the multiple sample processed images and the multiple sample defect information, and then imports the training set into the training model for model training based on a preset training model to construct a defect detection model.

[0066] Specifically, Fig. 9 A structural block diagram of the detection model training unit 102 in the system provided in an embodiment of the present invention is shown.

[0067] Among them, in the preferred implementation manner provided by the present invention, the detection model training unit 102 specifically includes: The defect analysis module 1021 is used to perform defect analysis on the plurality of sample processing images based on the sample marking information, and determine the sample defect types and sample defect locations of the plurality of sample processing images; An information generating module 1022, configured to generate sample defect information of a plurality of sample processed images according to a plurality of sample defect types and a plurality of sample defect positions; A training set construction module 1023, configured to construct a training set according to the plurality of sample processed images and the plurality of corresponding sample defect information; The model building module 1024 is used to import the model training to the training set based on the preset training model to build a defect detection model.

[0068] Furthermore, the electronic component surface defect detection system based on artificial intelligence also includes: The detection model optimization unit 103 is used to obtain a plurality of verification processing images and verification mark information during a preset model verification period, verify and optimize the defect detection model, and generate an optimized detection model.

[0069] In an embodiment of the present invention, during a preset model verification period, the detection model optimization unit 103 creates a batch verification task, obtains a plurality of verification processing images and verification marking information according to the batch verification task, obtains a plurality of verification detection results exported by the defect detection model by importing the plurality of verification processing images into the defect detection model, and then performs verification comparison on the plurality of verification detection results according to the verification marking information, screens a plurality of verification error results, and screens a plurality of verification error images and corresponding marking information according to the plurality of verification error results, performs optimization training on the defect detection model according to the plurality of verification error images and the corresponding marking information, and generates an optimized detection model.

[0070] Specifically, Fig.10 It shows a structural block diagram of the detection model optimization unit 103 in the system provided by an embodiment of the present invention.

[0071] Among them, in the preferred implementation manner provided by the present invention, the detection model optimization unit 103 specifically includes: The task creation module 1031 is used to create batch verification tasks in a preset model verification period; The task processing module 1032 is used to obtain a plurality of verification processing images and verification mark information according to the batch verification task; A verification import module 1033 is used to import the multiple verification processed images into the defect detection model to obtain multiple verification detection results exported by the defect detection model; A verification comparison module 1034 is used to perform verification comparison on the multiple verification test results according to the verification mark information, and filter multiple verification error results; An error screening module 1035 is used to screen a plurality of verification error images and corresponding marking information according to the plurality of verification error results; The model optimization module 1036 is used to optimize the defect detection model according to the multiple verification failure images and the corresponding marking information to generate an optimized detection model.

[0072] Furthermore, the electronic component surface defect detection system based on artificial intelligence also includes: The detection shooting processing unit 104 is used to acquire the detection shooting images in real time based on artificial intelligence technology, and process the detection shooting images to generate detection processing images.

[0073] In an embodiment of the present invention, the detection and shooting processing unit 104 directly arranges the optimized detection model in the production line system. During the production and processing of electronic components, based on artificial intelligence technology, a real-time detection task is created. According to the real-time detection task, the detection and shooting images are acquired in real time. By performing image standardization, image denoising and image enhancement processing on the detection and shooting images, a detection processing image is generated.

[0074] The real-time defect detection unit 105 is used to import the inspection and processing image into the optimized inspection model, perform real-time defect detection on the inspection and processing image, and generate and display defect detection results.

[0075] In an embodiment of the present invention, the real-time defect detection unit 105 imports the detection and processing image into the optimized detection model, obtains the defect detection result derived from the detection by the optimized detection model, obtains the result display address, and then transmits and displays the defect detection result according to the result display address.

[0076] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0077] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0078] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An electronic component surface defect detection method based on artificial intelligence, characterized in that: The method specifically comprises the following steps: Photographing a plurality of electronic component samples to obtain a plurality of sample photographed images, and processing the plurality of sample photographed images to generate a plurality of sample processed images, and obtaining sample marking information; Based on the sample marking information, defect feature analysis is performed on the plurality of sample processed images, and based on a preset training model, a defect detection model is trained and constructed; In a preset model verification period, a plurality of verification processing images and verification mark information are obtained, the defect detection model is verified and optimized, and an optimized detection model is generated; Based on artificial intelligence technology, the detection image is acquired in real time, and the detection image is processed to generate a detection processing image; The inspection processed image is imported into the optimized inspection model, real-time defect detection is performed on the inspection processed image, and defect detection results are generated and displayed.

2. The method for detecting surface defects of electronic components based on artificial intelligence according to claim 1, characterized in that: The method of photographing a plurality of electronic component samples to obtain a plurality of sample photographed images, processing the plurality of sample photographed images to generate a plurality of sample processed images, and obtaining sample marking information specifically includes the following steps: Get device shooting parameters; According to the device shooting parameters, a plurality of electronic component samples are photographed to obtain a plurality of sample photographed images; Performing image standardization, image denoising and image enhancement processing on the plurality of sample captured images to generate a plurality of sample processed images; Get sample label information.

3. The method for detecting surface defects of electronic components based on artificial intelligence according to claim 2, characterized in that: Performing image standardization, image denoising and image enhancement processing on the plurality of sample captured images to generate a plurality of sample processed images, the specific steps are as follows: Extract sample pixels, brightness values ​​of pixels and original pixel values ​​from sample captured images; Based on the brightness value of each pixel, the brightness mean of the sample captured image is calculated; Based on the brightness value of each pixel, the standard deviation of the brightness value is calculated; Subtract the brightness mean of the sample image from the brightness value of each pixel, and then divide it by the standard deviation of the brightness value to obtain the standardized brightness value; Assign the standardized brightness value to the sample pixel to generate a standardized image; Determine the denoising weight through the Gaussian kernel function; Extracting standardized pixels from the standardized image; For each standardized pixel, the weighted average of the pixels around the current standardized pixel is calculated using the denoising weight, and then the weighted average of the pixels around the current standardized pixel is assigned to the standardized pixel to generate a denoised image; Extract denoised pixels from the denoised image; Based on the denoised pixel, the Laplace value of the pixel is obtained by calculation, and the Laplace value is added to the original pixel value to obtain the enhanced value of the pixel; Assign the enhanced value of the pixel to the denoised pixel to generate an enhanced image; Determining sample processing weights based on the standardized image, the denoised image, and the enhanced image; A sample processing image is generated according to the standardized brightness value, the weighted average of the pixels around the current standardized pixel, and the Laplace value of the pixel, combined with the sample processing weight.

4. The method for detecting surface defects of electronic components based on artificial intelligence according to claim 3, characterized in that: The defect feature analysis is performed on the plurality of sample processed images based on the sample marking information, and the training and construction of the defect detection model based on the preset training model specifically includes the following steps: Based on the sample marking information, performing defect analysis on the plurality of sample processing images to determine sample defect types and sample defect locations of the plurality of sample processing images; Generating sample defect information of a plurality of sample processed images according to a plurality of sample defect types and a plurality of sample defect positions; Constructing a training set according to the plurality of sample processed images and the plurality of corresponding sample defect information; Based on the preset training model, the training set is imported into the model training to construct a defect detection model.

5. The method for detecting surface defects of electronic components based on artificial intelligence according to claim 4, characterized in that: The method of obtaining a plurality of verification processing images and verification mark information during a preset model verification period, verifying and optimizing the defect detection model, and generating an optimized detection model specifically includes the following steps: Create batch verification tasks during the preset model verification period; According to the batch verification task, a plurality of verification processing images and verification mark information are obtained; Importing the plurality of verification processed images into the defect detection model, and obtaining a plurality of verification detection results derived from the defect detection model; According to the verification mark information, verify and compare the multiple verification test results to screen multiple verification error results; According to the plurality of verification failure results, screening a plurality of verification failure images and corresponding marking information; The defect detection model is optimized and trained according to the plurality of verification failure images and the corresponding marking information to generate an optimized detection model.

6. The method for detecting surface defects of electronic components based on artificial intelligence according to claim 5, characterized in that: According to the verification mark information, multiple verification test results are verified and compared, and multiple verification error results are screened. The specific steps are as follows: Acquire real defect information of the sample, wherein the real defect information of the sample includes the sample defect type and the sample defect position; Associating each verification processing image, verification mark information, and verification test result to obtain a set of paired verification data; wherein each set of verification data includes a verification processing image, defect information, and verification test result; Determine whether the sample defect type predicted by the model is consistent with the sample defect type in the actual defect information of the sample. If not, record the classification error value accordingly; Calculate and compare the intersection-and-union ratio of the sample defect position predicted by the model and the sample defect position in the actual defect information of the sample. If the intersection-and-union ratio is lower than the preset standard value, record the positioning error value; Based on the classification error value and the positioning error value, a weighted sum is performed in combination with a preset comprehensive difference weight to obtain a comprehensive difference value; Preset dynamic thresholds based on different sample defect types; The comprehensive difference value is compared with the dynamic threshold. If the comprehensive difference value exceeds the dynamic threshold, the target sample is judged as a verification error result, and a verification error result set is generated; wherein the error result set includes all samples that exceed the dynamic threshold and the verification processing images and verification mark information corresponding to the samples.

7. The method for detecting surface defects of electronic components based on artificial intelligence according to claim 6, characterized in that: The method of acquiring the detection image in real time based on artificial intelligence technology and processing the detection image to generate the detection processed image specifically includes the following steps: Create real-time detection tasks based on artificial intelligence technology; According to the real-time detection task, acquiring detection images in real time; The detection shot image is subjected to image standardization, image denoising and image enhancement processing to generate a detection processed image.

8. The method for detecting surface defects of electronic components based on artificial intelligence according to claim 7, characterized in that: The detection image is subjected to image standardization, image denoising and image enhancement processing to generate a detection processed image. The specific steps are as follows: Obtain the detection image and environmental parameters, where the environmental parameters include light intensity; Based on the detection shot image, calculate the pixel mean value of the detection shot image and the standard deviation of the detection shot image; Dynamically adjust the normalization parameters based on the pixel mean of the detected image, the standard deviation of the detected image, and the environmental parameters; Applying the standardized parameters to the detection image to generate a standardized detection image; Performing wavelet decomposition on the standardized detection shot image to obtain a detection shot image after wavelet decomposition; In the detection shooting image after wavelet decomposition, the non-local similarity block matching method is used to smooth the noise in the area where the texture details are retained to obtain the denoised detection shooting image; Build an attention map generation module containing a convolutional neural network; The denoised inspection image is input into the attention map generation module to obtain the attention map, and the key areas that need to be enhanced are marked in the attention map based on the sample defect location; Based on the attention weight, the pixel values ​​in the key area are nonlinearly stretched, and the brightness of the non-key area is suppressed to generate an enhanced detection image. The enhanced detection captured image is adjusted to the output size and color space required by the optimized detection model, and finally the detection processed image is output.

9. The method for detecting surface defects of electronic components based on artificial intelligence according to claim 8, characterized in that: The step of importing the inspection processed image into the optimized inspection model, performing real-time defect detection on the inspection processed image, and generating and displaying defect detection results specifically includes the following steps: Importing the detected processed image into the optimized detection model; Obtaining defect detection results derived from detection using the optimized detection model; Get the result display address; Transmitting and displaying the defect detection result according to the result display address; The specific steps of obtaining the defect detection results derived from the optimized detection model are as follows: Input the inspection processed image into the optimized inspection model, analyze the potential defect area in the inspection processed image to generate an original monitoring result set; wherein the original monitoring result set includes a data list of detection frame coordinates, confidence scores, and defect category labels; Generate a dynamic category weight list based on sample defect types; The original monitoring result set is screened according to the sample defect type to generate a target defect subset; Based on historical data, dynamically modified weight values ​​are extracted from the dynamic category weight list; Based on the dynamic correction weight value and the confidence score in the original monitoring result set, a corrected confidence score is obtained through weighted calculation, and the corrected confidence score is returned to the original monitoring result set to generate a corrected detection result set; wherein the corrected detection result set includes a data list of detection box coordinates, corrected confidence scores, and defect category labels; Setting a baseline threshold based on the training data used during the training of the defect detection model; The average value of sample failure rate is calculated through historical data; The dynamic threshold is limited according to the average value of the reference threshold and the unqualified rate to obtain the dynamic judgment threshold; extracting revised confidence scores from the revised set of detection results, and comparing each revised confidence score with a dynamic threshold; Performing confidence attenuation processing on the corrected confidence scores that do not reach the dynamic threshold to obtain attenuated confidence scores; The attenuated confidence scores are filtered to generate and export defect detection results; wherein the defect detection results include a data list of detection frame coordinates, attenuated confidence scores, and defect category labels.

10. The electronic component surface defect detection system based on artificial intelligence is characterized by: The system applies the electronic component surface defect detection method based on artificial intelligence as described in any one of claims 1 to 9 above, and the system includes a sample shooting processing unit, a detection model training unit, a detection model optimization unit, a detection shooting processing unit and a real-time defect detection unit, wherein: A sample shooting and processing unit, used to shoot a plurality of electronic component samples, obtain a plurality of sample shooting images, process the plurality of sample shooting images, generate a plurality of sample processing images, and obtain sample marking information; A detection model training unit, used to perform defect feature analysis on a plurality of sample processed images based on the sample label information, and train and construct a defect detection model based on a preset training model; A detection model optimization unit, used to obtain a plurality of verification processing images and verification mark information during a preset model verification period, verify and optimize the defect detection model, and generate an optimized detection model; A detection shooting processing unit, used to acquire detection shooting images in real time based on artificial intelligence technology, and process the detection shooting images to generate detection processing images; A real-time defect detection unit is used to import the inspection and processing image into the optimized inspection model, perform real-time defect detection on the inspection and processing image, and generate and display defect detection results.

Images