Part defect detection method and system based on computer vision

Abstract

The embodiment of the invention relates to the technical field of part defect detection, and particularly discloses a part defect detection method and system based on computer vision. According to the embodiment of the invention, production monitoring of the camera module is carried out; when the defect detection scene is met, carrying out initial detection shooting, and carrying out substrate edge analysis; when the edge stability defect does not exist, detecting the center point of the substrate; and when the suspicious center deviation exists, contrast shooting and defect contrast detection analysis are carried out. When a defect detection scene is met, a current production module is controlled to carry out initial detection shooting control, substrate edge analysis is carried out on an initial detection shooting image, substrate center point detection is carried out when no edge stable defect exists, and when suspicious center deviation exists, a standard shooting module is selected to carry out comparison shooting and defect comparison detection analysis. Therefore, the camera module is directly used for detection shooting, and effective defect detection can be carried out on conditions such as substrate placement and central point deviation.

Description

Technical Field

[0001] The present invention belongs to the technical field of part defect detection, and particularly relates to a part defect detection method and system based on computer vision. Background Art

[0002] Part defect detection mainly refers to the process of detecting various defects existing on the surface or inside of parts, which is an important link to ensure part quality and production safety. Usually, advanced detection technologies such as machine vision detection, X-ray non-destructive detection, and chemical composition analysis are used to achieve comprehensive and accurate detection of parts.

[0003] In the prior art, for part defect detection based on computer vision, it is usually necessary to additionally set up a camera for shooting. During the production process of the camera module, it is impossible to use the camera module itself for detection shooting, and it is unable to effectively detect defects such as the placement of the substrate and the deviation of the center point. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a part defect detection method and system based on computer vision, aiming to solve the problems proposed in the background art.

[0005] To achieve the above purpose, the embodiments of the present invention provide the following technical solutions: A part defect detection method based on computer vision, the method specifically includes the following steps: Conduct production monitoring of the camera module, obtain production monitoring data, analyze the production monitoring data, and determine whether it meets the defect detection scenario; When the defect detection scenario is met, perform initial inspection shooting control, obtain an initial inspection shooting image, and analyze the substrate edge of the initial inspection shooting image to determine whether there is an edge stability defect; When there is an edge stability defect, give an edge defect alarm; When there is no edge stability defect, detect the center point of the substrate of the initial inspection shooting image to determine whether there is a suspected center deviation; When there is a suspected center deviation, select a standard camera module, perform comparison shooting control, obtain a comparison shooting image, and conduct defect comparison detection analysis to output comparison detection information.

[0006] As a further limitation of the technical solution of the embodiments of the present invention, the conduct production monitoring of the camera module, obtain production monitoring data, analyze the production monitoring data, and determine whether it meets the defect detection scenario specifically includes the following steps: Determine the target module production line; Conduct production monitoring of the camera module on the target module production line to obtain production monitoring data; Analyze the production monitoring data to determine the current production module; Conduct a production change analysis on the current production module to determine whether the defect detection scenario is met.

[0007] As a further limitation of the technical solution of the embodiment of the present invention, when the defect detection scenario is met, perform initial inspection shooting control, obtain an initial inspection shooting image, and conduct a substrate edge analysis on the initial inspection shooting image to determine whether there is an edge stability defect, which specifically includes the following steps: When the defect detection scenario is met, generate an initial inspection shooting instruction; Send the initial inspection shooting instruction to the current production module, and perform an initial inspection shooting through the current production module to obtain an initial inspection shooting image; Conduct a substrate edge analysis on the initial inspection shooting image to obtain substrate edge length data; Compare the lengths of the corresponding edges of the substrate edge length data to determine whether there is an edge stability defect.

[0008] As a further limitation of the technical solution of the embodiment of the present invention, when there is an edge stability defect, perform an edge defect alarm, which specifically includes the following steps: When there is an edge stability defect, generate an edge defect signal; Send the edge defect signal to multiple preset defect alarm addresses; Respond to the edge defect signal to perform an edge defect alarm.

[0009] As a further limitation of the technical solution of the embodiment of the present invention, when there is no edge stability defect, conduct a substrate center point detection on the initial inspection shooting image to determine whether there is a suspected center deviation, which specifically includes the following steps: When there is no edge stability defect, generate a center point detection instruction; According to the center point detection instruction, conduct a substrate center point detection on the initial inspection shooting image to obtain center point detection data; Identify the module center point and the substrate center point from the center point detection data; Compare the module center point with the substrate center point according to a preset center point error range, and record the center point comparison result; Judge whether there is a suspected center deviation according to the center point comparison result.

[0010] As a further limitation of the technical solution of the embodiment of the present invention, when there is a suspected center deviation, select a standard camera module, perform a comparison shooting control, obtain a comparison shooting image, and conduct a defect comparison detection analysis to output comparison detection information, which specifically includes the following steps: When there is a suspected center deviation, select a standard camera module according to the current production module; Generate a comparison shooting instruction; Send the comparison shooting instruction to the standard camera module, perform comparison shooting through the standard camera module, and obtain a comparison shooting image; Perform defect comparison detection and analysis on the comparison shooting image and the initial inspection shooting image, and output comparison detection information.

[0011] A part defect detection system based on computer vision, the system includes a production monitoring and processing unit, a substrate edge analysis unit, an edge defect alarm unit, a substrate center point detection unit, and a defect comparison detection unit, wherein: The production monitoring and processing unit is used to monitor the production of the camera module, obtain production monitoring data, analyze the production monitoring data, and judge whether it meets the defect detection scenario; The substrate edge analysis unit is used to control the initial inspection shooting when the defect detection scenario is met, obtain the initial inspection shooting image, and analyze the substrate edge of the initial inspection shooting image to judge whether there is an edge stability defect; The edge defect alarm unit is used to alarm the edge defect when there is an edge stability defect; The substrate center point detection unit is used to detect the substrate center point of the initial inspection shooting image when there is no edge stability defect, and judge whether there is a suspected center deviation; The defect comparison detection unit is used to select a standard camera module when there is a suspected center deviation, control the comparison shooting, obtain the comparison shooting image, and perform defect comparison detection and analysis, and output comparison detection information.

[0012] As a further limitation of the technical solution of the embodiment of the present invention, the production monitoring and processing unit specifically includes: The production line determination module is used to determine the target module production line; The production monitoring module is used to monitor the production of the camera module on the target module production line and obtain production monitoring data; The module determination module is used to analyze the production monitoring data and determine the current production module; The scenario judgment module is used to analyze the production changes of the current production module and judge whether it meets the defect detection scenario.

[0013] As a further limitation of the technical solution of the embodiment of the present invention, the substrate edge analysis unit specifically includes: The initial inspection instruction generation module is used to generate an initial inspection shooting instruction when the defect detection scenario is met; The initial inspection shooting module is used to send the initial inspection shooting instruction to the current production module, perform initial inspection shooting through the current production module, and obtain an initial inspection shooting image; The edge analysis module is used to analyze the substrate edge of the initial inspection shooting image and obtain substrate edge length data; The length comparison module is used to compare the lengths of the corresponding edges of the substrate edge length data to determine whether there is an edge stability defect.

[0014] As a further limitation of the technical solution of the embodiment of the present invention, the substrate center point detection unit specifically includes: The center point detection instruction generation module is used to generate a center point detection instruction when there is no edge stability defect; The substrate center point detection module is used to detect the substrate center point of the initial inspection shooting image according to the center point detection instruction and obtain center point detection data; The center point recognition module is used to identify the module center point and the substrate center point from the center point detection data; The center point comparison module is used to compare the module center point with the substrate center point according to a preset center point error range and record the center point comparison result; The center deviation judgment module is used to judge whether there is a suspected center deviation according to the center point comparison result.

[0015] Compared with the prior art, the beneficial effects of the present invention are: In the embodiment of the present invention, production monitoring of the camera module is carried out; when the defect detection scenario is met, initial inspection shooting is performed and substrate edge analysis is carried out; when there is no edge stability defect, substrate center point detection is carried out; when there is a suspected center deviation, comparison shooting and defect comparison detection analysis are carried out. When the defect detection scenario is met, the current production module can be controlled to perform initial inspection shooting control, and the substrate edge of the initial inspection shooting image can be analyzed. When there is no edge stability defect, substrate center point detection is carried out. When there is a suspected center deviation, a standard camera module is selected for comparison shooting and defect comparison detection analysis, so as to directly use the camera module itself for detection shooting, and effective defect detection of the placement of the substrate, center point offset, etc. can be realized. Description of the Drawings

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0017] Figure 1 Shows the flowchart of the method provided by the embodiment of the present invention.

[0018] Figure 2 Shows the flowchart of defect detection scenario judgment in the method provided by the embodiment of the present invention.

[0019] Figure 3 Shows the flowchart of edge stability defect judgment in the method provided by the embodiment of the present invention.

[0020] Figure 4 Shows the flowchart of performing edge defect alarm in the method provided by the embodiment of the present invention.

[0021] Figure 5 Shows the flowchart of suspicious center deviation judgment in the method provided by the embodiment of the present invention.

[0022] Figure 6 Shows the flowchart of performing defect comparison detection in the method provided by the embodiment of the present invention.

[0023] Figure 7 Shows the application architecture diagram of the system provided by the embodiment of the present invention.

[0024] Figure 8 Shows the structural block diagram of the production monitoring and processing unit in the system provided by the embodiment of the present invention.

[0025] Figure 9 Shows the structural block diagram of the substrate edge analysis unit in the system provided by the embodiment of the present invention.

[0026] Figure 10 Shows the structural block diagram of the substrate center point detection unit in the system provided by the embodiment of the present invention. Detailed implementation manners

[0027] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0028] It can be understood that in the prior art, for part defect detection based on computer vision, an additional camera usually needs to be set up for shooting. During the production process of the camera module, it is impossible to use the camera module itself for detection and shooting, and it is impossible to effectively detect defects such as the placement of the substrate and the deviation of the center point.

[0029] To solve the above problems, the embodiment of the present invention performs production monitoring of the camera module, obtains production monitoring data, analyzes the production monitoring data, and determines whether the defect detection scenario is met; when the defect detection scenario is met, performs initial inspection shooting control, obtains the initial inspection shooting image, and performs substrate edge analysis on the initial inspection shooting image to determine whether there is an edge stability defect; when there is an edge stability defect, performs an edge defect alarm; when there is no edge stability defect, performs substrate center point detection on the initial inspection shooting image to determine whether there is a suspicious center deviation; when there is a suspicious center deviation, selects a standard camera module, performs comparative shooting control, obtains comparative shooting images, and performs defect comparative detection analysis, and outputs comparative detection information. When the defect detection scenario is met, the current production module can be controlled to perform initial inspection shooting control, and substrate edge analysis can be performed on the initial inspection shooting image. When there is no edge stability defect, substrate center point detection is performed. When there is a suspicious center deviation, the standard camera module is selected to perform comparative shooting and defect comparative detection analysis, thereby directly using the camera module itself for detection and shooting, and effective defect detection can be achieved for substrate placement, center point offset, and other situations.

[0030] Figure 1 A flow chart of a method provided by an embodiment of the present invention is shown.

[0031] Specifically, the method for detecting part defects based on computer vision includes the following steps: Step S101: perform production monitoring of the camera module, obtain production monitoring data, analyze the production monitoring data, and determine whether the defect detection scenario is met.

[0032] In an embodiment of the present invention, by determining the target module production line, the production monitoring of the camera module of the target module production line is performed, the production monitoring data is obtained, and then the production monitoring data is analyzed to determine the current production module and the previous production module. The current production module is compared with the previous production module to determine whether there is a change in module production. When there is a change in module production, it is determined that the defect detection scenario is met; and when there is no change in module production, it is determined that the defect detection scenario is not met.

[0033] Specifically, Figure 2 A flow chart of defect detection scenario judgment in the method provided by an embodiment of the present invention is shown.

[0034] Among them, in the preferred embodiment provided by the present invention, the production monitoring of the camera module, obtaining production monitoring data, analyzing the production monitoring data, and determining whether the defect detection scenario is met specifically include the following steps: Step S1011, determining the target module production line; Step S1012, perform production monitoring of camera modules on the target module production line to obtain production monitoring data; Step S1013, analyze the production monitoring data to determine the currently produced module; Step S1014, perform production change analysis on the currently produced module to determine whether the defect detection scenario is met.

[0035] Specifically, the steps for performing production change analysis on the currently produced module to determine whether the defect detection scenario is met are as follows: Obtain process parameters from the production detection data; Perform time series sampling on each process parameter, and record the parameter values at the current moment and the previous two sampling points adjacent to the current moment; Take the first derivative of the parameter values at the current moment and the previous two sampling points adjacent to the current moment to obtain the change rate of the process parameter; Based on the production situation of camera modules of the same model in the past, determine the dynamic weight value of the process parameter; Multiply the change rate of each process parameter by the dynamic weight value of the corresponding process parameter to obtain the weighted value of all process parameters; Perform a summation calculation on the weighted values of all process parameters to obtain the weighted summation value of the process parameters; Determine the adaptive coefficient according to the target module production line; Multiply the weighted summation value of the process parameters by the adaptive coefficient to obtain the evaluation value of the process parameter change; Obtain the current production duration and production standard cycle through the target module production line, and obtain the time difference in production based on the current production duration and production standard cycle; Use an exponential function to perform non-linear conversion on the time difference in production to generate a time sensitivity factor for production; Collect multi-spectral images and a standard optical feature library through the camera module, extract optical feature vectors in multiple dimensions from the multi-spectral images, and obtain standard optical feature vectors from the standard feature library; Based on the optical feature vectors and standard optical feature vectors, calculate the Euclidean distance value of optics in the feature space; Multiply the Euclidean distance value of optics by the time sensitivity factor for production to obtain the evaluation value of the optical feature difference; Linearly superimpose the evaluation value of the process parameter change and the evaluation value of the optical feature difference to obtain the comprehensive evaluation value of production; Determine the analysis threshold based on the production standard; Perform production change analysis on the currently produced module based on the comprehensive evaluation value of production and the analysis threshold to obtain a production change analysis report; Judge whether the defect detection scenario is satisfied based on the production change analysis report.

[0036] Furthermore, the computer vision-based part defect detection method further includes the following steps: Step S102, when the defect detection scenario is satisfied, perform initial inspection shooting control, obtain an initial inspection shooting image, and perform substrate edge analysis on the initial inspection shooting image to determine whether there is an edge stability defect.

[0037] In an embodiment of the present invention, when the defect detection scenario is satisfied, an initial inspection shooting instruction is generated and sent to the current production module. The current production module performs initial inspection shooting to obtain an initial inspection shooting image. Then, substrate edge analysis is performed on the initial inspection shooting image to obtain substrate edge length data. By comparing and analyzing the opposite side lengths of the substrate edge length data, it is determined whether the opposite side lengths of the substrate edge are equal. When the opposite side lengths of the substrate edge are not equal, it is determined that there are edge stability defects such as substrate warping and substrate offset; when the opposite side lengths of the substrate edge are equal, it is determined that there is no edge stability defect.

[0038] Specifically, Figure 3 The flowchart of edge stability defect judgment in the method provided by the embodiment of the present invention is shown.

[0039] Among them, in the preferred embodiment provided by the present invention, the steps of performing initial inspection shooting control, obtaining an initial inspection shooting image, and performing substrate edge analysis on the initial inspection shooting image to determine whether there is an edge stability defect when the defect detection scenario is satisfied specifically include the following steps: Step S1021, when the defect detection scenario is satisfied, generate an initial inspection shooting instruction; Step S1022, send the initial inspection shooting instruction to the current production module, and the current production module performs initial inspection shooting to obtain an initial inspection shooting image; Step S1023, perform substrate edge analysis on the initial inspection shooting image to obtain substrate edge length data, obtain substrate material attribute parameters, the environmental temperature of the production line, and the process standard database; Step S1024, compare the lengths of the corresponding edges of the substrate edge length data in combination with the substrate material attribute parameters, the environmental temperature of the production line, and the process standard database to determine whether there is an edge stability defect.

[0040] Specifically, the steps of comparing the lengths of the corresponding edges of the substrate edge length data in combination with the substrate material attribute parameters, the environmental temperature of the production line, and the process standard database to determine whether there is an edge stability defect are as follows: Obtain the real-time working condition temperature value through the environmental temperature of the production line, and obtain the material expansion coefficient, material hardness value, and ductility value from the substrate attribute parameters; Retrieve the curvature stability threshold from the process standard database; Calculate based on the material expansion coefficient and the real-time working condition temperature value to obtain the dynamic length tolerance threshold; Calculate based on the material hardness value and the ductility value to obtain the length balance factor; Obtain the measured edge length and the standard theoretical length from the substrate edge length data. By comparing each measured edge length with the standard theoretical length, obtain the absolute deviation value; Normalize the absolute deviation value using the dynamic length tolerance threshold to obtain the normalized absolute deviation value; Adjust the weight of the normalized absolute deviation value using the length balance factor to obtain the standardized length deviation value; Obtain the edge point coordinates from the substrate edge length data, and based on the edge point coordinates, use the five-point division method to obtain the curvature change rate; Perform discretization processing and maximum value extraction on the curvature change rate in sequence to obtain the maximum mutation eigenvalue. Use the curvature stability threshold to standardize the maximum mutation eigenvalue to obtain the standardized mutation eigenvalue; Perform reverse weight adjustment on the standardized mutation eigenvalue using the length balance factor to obtain the scoring value of curvature stability; Add the standardized length deviation value and the scoring value of curvature stability to obtain the segmented comprehensive scoring value, and perform arithmetic averaging on the segmented comprehensive scoring value to obtain the average value of the comprehensive scoring; Determine the gradient classification for judging edge stability defects, and combine the average value of the comprehensive scoring to judge the edge stability defects of the substrate.

[0041] Furthermore, the computer vision-based part defect detection method further includes the following steps: Step S103, when there is an edge stability defect, perform edge defect alarm.

[0042] In the embodiment of the present invention, in the case of having an edge stability defect, generate an edge defect signal, and then transmit the edge defect signal according to multiple preset defect alarm addresses, so as to be able to respond to the edge defect signal and perform edge defect alarm until after the repair of the edge stability defect is completed, stop the edge defect alarm.

[0043] Specifically, Figure 4 Shows the flowchart of performing edge defect alarm in the method provided by the embodiment of the present invention.

[0044] Among them, in the preferred embodiment provided by the present invention, when there is an edge stability defect, performing an edge defect alarm specifically includes the following steps: Step S1031, when there is an edge stability defect, generating an edge defect signal; Step S1032, sending the edge defect signal to a plurality of preset defect alarm addresses; Step S1033: responding to the edge defect signal and issuing an edge defect alarm.

[0045] Specifically, when there is an edge stability defect, an edge defect signal is generated, wherein the accuracy of signal generation is improved by the edge defect determination coefficient. The expression of the edge defect determination coefficient is: ; in, represents the edge defect determination coefficient, represents the spatial weight matrix, represents the gradient amplitude, Indicates the current pixel angle, Indicates the base angle, represents the environmental adaptation factor, Represents the high-frequency noise energy value, represents the local standard deviation, represents the total number of spatial neighborhood pixels, Indicates the index of the neighboring pixel.

[0046] Furthermore, the computer vision-based part defect detection method further includes the following steps: Step S104 : When there is no edge stability defect, performing substrate center point detection on the initial inspection image to determine whether there is a suspicious center deviation.

[0047] In an embodiment of the present invention, in the absence of edge stability defects, a center point detection instruction is generated, and then the substrate center point detection is performed on the initial inspection image according to the center point detection instruction to obtain center point detection data. The module center point and the substrate center point are determined by performing center point identification on the center point detection data. Then, according to a preset center point error range, the module center point and the substrate center point are compared, and the center point comparison result is recorded. Then, according to the center point comparison result, when the distance between the module center point and the substrate center point is greater than the center point error range, it is determined that there is a suspicious center deviation; and when the distance between the module center point and the substrate center point is not greater than the center point error range, it is determined that there is no suspicious center deviation.

[0048] Specifically, Figure 5 The flowchart of the suspicious center deviation judgment in the method provided by the embodiment of the present invention is shown.

[0049] Among them, in the preferred embodiment provided by the present invention, when there is no edge stability defect, performing substrate center point detection on the initial inspection image to determine whether there is suspicious center deviation specifically includes the following steps: Step S1041: When there is no edge stability defect, a center point detection instruction is generated; Step S1042: performing substrate center point detection on the initial inspection image according to the center point detection instruction to obtain center point detection data; Step S1043, identifying the module center point and the substrate center point from the center point detection data; Step S1044, comparing the module center point with the substrate center point according to a preset center point error range, and recording the center point comparison result; Step S1045 : judging whether there is a suspicious center deviation according to the center point comparison result.

[0050] Specifically, according to a preset center point error range, the center point of the module is compared with the center point of the substrate, and the center point comparison result is recorded. The specific steps are as follows: Determine the three-dimensional coordinates of the module center point by identifying the module center point; Extract and generate the three-dimensional coordinate components of the center point from the three-dimensional coordinates of the center point of the module; Based on the material expansion coefficient and the real-time working temperature value, the dynamic tolerance threshold is calculated; Obtain substrate deformation factor from module production line; A three-dimensional weighted deviation value is calculated based on the three-dimensional coordinate components of the center point and the dynamic tolerance threshold, and the three-dimensional weighted deviation values are accumulated and summed to obtain a total three-dimensional weighted deviation value; Obtain standard temperature values from the process standard database; The absolute temperature difference is calculated based on the real-time working temperature value and the standard temperature value; The absolute difference of temperature is selected by machine through the exponential function decay model to obtain the nonlinear temperature compensation coefficient; Multiply the substrate deformation factor and the nonlinear temperature compensation coefficient to obtain a joint correction value; The total value of the three-dimensional weighted deviation is added to the joint correction value to obtain the composite deviation index; Determine the deviation judgment threshold according to the preset center point error range; The composite deviation index is compared with the deviation judgment threshold, and the comparison result is generated and recorded.

[0051] Furthermore, the computer vision-based part defect detection method further includes the following steps: Step S105: When there is a suspected center deviation, select a standard camera module, perform contrast shooting control, obtain contrast shooting images, and perform defect contrast detection and analysis to output contrast detection information.

[0052] In an embodiment of the present invention, in the case of a suspected center deviation, according to the current production module, a standard camera module is selected. After the installation of the standard camera module is completed, a contrast shooting instruction is generated. By sending the contrast shooting instruction to the standard camera module, the standard camera module performs contrast shooting to obtain contrast shooting images. Then, the contrast shooting images are subjected to defect contrast detection and analysis with the initial inspection shooting images to output contrast detection information. Among them, if the module center point of the contrast shooting image and the initial inspection shooting image is consistent with the substrate center point, it is determined that the substrate center point has an offset defect; if the module center point of the contrast shooting image and the initial inspection shooting image is inconsistent with the substrate center point, it is determined that the substrate center point is normal, and the current production module has quality defects.

[0053] Specifically, Figure 6 The flowchart of performing defect contrast detection in the method provided by the embodiment of the present invention is shown.

[0054] Among them, in the preferred embodiment provided by the present invention, the steps of selecting a standard camera module, performing contrast shooting control, obtaining contrast shooting images, and performing defect contrast detection and analysis to output contrast detection information when there is a suspected center deviation specifically include the following steps: Step S1051: When there is a suspected center deviation, select a standard camera module according to the current production module; Step S1052: Generate a contrast shooting instruction; Step S1053: Send the contrast shooting instruction to the standard camera module, and perform contrast shooting through the standard camera module to obtain contrast shooting images. Detect the current production module through a light intensity sensor to obtain the current ambient light intensity value, and obtain the roughness parameter of the part surface through a surface profiler; Step S1054: Combine the current ambient light intensity value and the roughness parameter, and perform defect contrast detection and analysis on the contrast shooting images and the initial inspection shooting images to output contrast detection information.

[0055] Specifically, combining the current ambient light intensity value and the roughness parameter, performing defect contrast detection and analysis on the contrast shooting images and the initial inspection shooting images to output contrast detection information, the specific steps are as follows: Obtain a test image from the contrast shooting image, obtain a standard image from the initial inspection shooting image, and determine the roughness quantization value through the roughness parameter; Use wavelet transform to process the test image and the standard image to generate a test feature map and a standard feature map; Extract the high-frequency detail components and low-frequency contour components for each feature level in the test feature map and the standard feature map to obtain the processed feature map; Perform edge enhancement processing on the processed feature map to obtain wavelet feature maps of each layer; Based on the current ambient light intensity value, use the light intensity compensation mechanism to adjust the initial weight to obtain the dynamically adjusted weight parameter; Match the basic noise threshold according to the roughness quantization value; Calculate the absolute value difference from the standard feature map layer by layer through the wavelet feature maps of each layer to obtain the feature difference value; Divide the feature map difference value by the basic noise threshold of the corresponding level to obtain the feature difference value of each level Apply exponential amplification processing to the feature difference values of each level to obtain the normalized and amplified feature difference values of each level; Use gamma correction to process the test image and the standard image to generate the brightness histogram of the test image and the brightness histogram of the standard image; Calculate the probability distribution difference value through the brightness histogram of the test image and the brightness histogram of the standard image; Use the smoothing factor to process the zero-probability region in the probability distribution difference value to obtain the brightness distribution difference quantization value; Multiply the normalized and amplified feature difference values of each level by the dynamically adjusted weight parameter to obtain the multi-scale feature fusion difference value; Multiply the brightness distribution difference quantization value by the dynamically adjusted weight parameter to obtain the light robustness difference factor; Perform weighted summation on the multi-scale feature fusion difference value and the light robustness difference factor to obtain the comprehensive defect contrast index; Input the comprehensive defect contrast index into the defect classifier to obtain the defect classification result, and further confirm the comparison detection information based on the defect classification result. Further, Figure 7 The application architecture diagram of the system provided by the embodiment of the present invention is shown.

[0056] Among them, in another preferred embodiment provided by the present invention, the part defect detection system based on computer vision includes: The production monitoring processing unit 101 is used to perform production monitoring of the camera module, obtain production monitoring data, analyze the production monitoring data, and judge whether it meets the defect detection scenario.

[0057] In an embodiment of the present invention, the production monitoring and processing unit 101 determines the target module production line, monitors the production of the camera module on the target module production line to obtain production monitoring data, analyzes the production monitoring data to determine the current production module and the previous production module, compares the current production module with the previous production module to determine whether there is a module production change, and determines that the defect detection scenario is satisfied when there is a module production change; while when there is no module production change, it is determined that the defect detection scenario is not satisfied.

[0058] Specifically, Figure 8 FIG. shows the structural block diagram of the production monitoring and processing unit 101 in the system provided by the embodiment of the present invention.

[0059] Among them, in the preferred embodiment provided by the present invention, the production monitoring and processing unit 101 specifically includes: A production line determination module 1011 for determining the target module production line; A production monitoring module 1012 for monitoring the production of the camera module on the target module production line to obtain production monitoring data; A module determination module 1013 for analyzing the production monitoring data to determine the current production module; A scenario judgment module 1014 for analyzing the production change of the current production module to determine whether the defect detection scenario is satisfied.

[0060] Furthermore, the part defect detection system based on computer vision further includes: A substrate edge analysis unit 102 for performing preliminary inspection shooting control when the defect detection scenario is satisfied, obtaining a preliminary inspection shooting image, and analyzing the substrate edge of the preliminary inspection shooting image to determine whether there is an edge stability defect.

[0061] In an embodiment of the present invention, when the defect detection scenario is satisfied, the substrate edge analysis unit 102 generates a preliminary inspection shooting instruction, sends the preliminary inspection shooting instruction to the current production module, performs preliminary inspection shooting through the current production module to obtain a preliminary inspection shooting image, then analyzes the substrate edge of the preliminary inspection shooting image to obtain substrate edge length data, and determines whether the opposite side lengths of the substrate edge are equal by comparing and analyzing the opposite side lengths of the substrate edge length data. When the opposite side lengths of the substrate edge are not equal, it is determined that there are edge stability defects such as substrate warping and substrate offset; while when the opposite side lengths of the substrate edge are equal, it is determined that there is no edge stability defect.

[0062] Specifically, Figure 9 FIG. shows the structural block diagram of the substrate edge analysis unit 102 in the system provided by the embodiment of the present invention.

[0063] In a preferred embodiment of the present invention, the substrate edge analysis unit 102 specifically includes: The initial inspection instruction generation module 1021 is used to generate an initial inspection shooting instruction when the defect detection scenario is met; The initial inspection shooting module 1022 is used to send the initial inspection shooting instruction to the current production module, perform initial inspection shooting through the current production module, and obtain the initial inspection shooting image; An edge analysis module 1023 is configured to perform substrate edge analysis on the initial inspection image to obtain substrate edge length data; The length comparison module 1024 is used to compare the length of the corresponding edge of the substrate edge length data to determine whether there is an edge stability defect.

[0064] Furthermore, the computer vision-based parts defect detection system also includes: The edge defect alarm unit 103 is used to issue an edge defect alarm when there is an edge stability defect.

[0065] In an embodiment of the present invention, in the case of an edge stability defect, the edge defect alarm unit 103 generates an edge defect signal, and then transmits the edge defect signal according to multiple preset defect alarm addresses, so as to respond to the edge defect signal and perform an edge defect alarm until the edge stability defect is repaired and the edge defect alarm is stopped.

[0066] The substrate center point detection unit 104 is used to perform substrate center point detection on the initial inspection image when there is no edge stability defect, and determine whether there is a suspicious center deviation.

[0067] In an embodiment of the present invention, in the absence of edge stability defects, the substrate center point detection unit 104 generates a center point detection instruction, and then performs substrate center point detection on the initial inspection image according to the center point detection instruction to obtain center point detection data. The module center point and the substrate center point are determined by performing center point identification on the center point detection data. Then, according to a preset center point error range, the module center point and the substrate center point are compared, and the center point comparison result is recorded. Then, according to the center point comparison result, when the distance between the module center point and the substrate center point is greater than the center point error range, it is determined that there is a suspicious center deviation; and when the distance between the module center point and the substrate center point is not greater than the center point error range, it is determined that there is no suspicious center deviation.

[0068] Specifically, Figure 10 FIG. 1 is a structural block diagram of the substrate center point detection unit 104 in the system provided by an embodiment of the present invention.

[0069] Among them, in the preferred embodiment provided by the present invention, the substrate center point detection unit 104 specifically includes: A center point detection instruction generation module 1041, configured to generate a center point detection instruction when there is no edge stability defect; A substrate center point detection module 1042, configured to perform substrate center point detection on the initial inspection captured image according to the center point detection instruction, and obtain center point detection data; A center point recognition module 1043, configured to recognize the module center point and the substrate center point from the center point detection data; A center point comparison module 1044, configured to compare the module center point with the substrate center point according to a preset center point error range, and record the center point comparison result; A center deviation judgment module 1045, configured to judge whether there is a suspicious center deviation according to the center point comparison result.

[0070] Furthermore, the computer vision-based part defect detection system further includes: A defect comparison detection unit 105, configured to select a standard camera module for contrast shooting control when there is a suspicious center deviation, obtain a contrast shooting image, and perform defect comparison detection analysis to output contrast detection information.

[0071] In the embodiment of the present invention, when there is a suspicious center deviation, the defect comparison detection unit 105 selects a standard camera module according to the current production module. After installing the standard camera module, a contrast shooting instruction is generated. By sending the contrast shooting instruction to the standard camera module, the standard camera module performs contrast shooting to obtain a contrast shooting image, and then the contrast shooting image and the initial inspection captured image are subjected to defect comparison detection analysis to output contrast detection information. Among them, if the module center point and the substrate center point of the contrast shooting image and the initial inspection captured image are consistent, it is determined that the substrate center point has an offset defect; if the module center point and the substrate center point of the contrast shooting image and the initial inspection captured image are inconsistent, it is determined that the substrate center point is normal, and the current production module has a quality defect.

[0072] It should be understood that, although the various steps in the flow chart of each embodiment of the present invention are shown in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in order, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0073] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When executed, the program can include the processes of the above-described method embodiments. Any reference to memory, storage, database, or other media used in the various embodiments provided herein may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0074] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0075] The above-described embodiments merely represent several implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation to the scope of the patent for the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several deformations and improvements can still be made, and these all fall within the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention shall be subject to the appended claims.

[0076] The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting part defects based on computer vision, characterized in that The method specifically includes the following steps: Perform production monitoring of the camera module, obtain production monitoring data, analyze the production monitoring data, and determine whether the defect detection scenario is met; When the defect detection scenario is met, perform preliminary inspection shooting control, obtain preliminary inspection shooting images, and perform substrate edge analysis on the preliminary inspection shooting images to determine whether there are edge stability defects; When there are edge stability defects, issue an edge defect alarm; When there are no edge stability defects, perform substrate center point detection on the preliminary inspection shooting images to determine whether there are suspicious center deviations; When there are suspicious center deviations, select a standard camera module, perform comparative shooting control, obtain comparative shooting images, and perform defect comparative detection analysis to output comparative detection information.

2. The method for detecting part defects based on computer vision according to claim 1, wherein The performing production monitoring of the camera module, obtaining production monitoring data, analyzing the production monitoring data, and determining whether the defect detection scenario is met specifically includes the following steps: Determine the target module production line; Perform production monitoring of the camera module on the target module production line to obtain production monitoring data; Analyze the production monitoring data to determine the current production module; Perform production change analysis on the current production module to determine whether the defect detection scenario is met.

3. The method for detecting part defects based on computer vision according to claim 2, characterized in that, The performing production change analysis on the current production module to determine whether the defect detection scenario is met, the specific steps are as follows: Obtain process parameters from the production detection data; Perform time series sampling on each process parameter, and record the parameter values at the current moment and the two adjacent previous sampling points to the current moment; Take the first derivative of the parameter values at the current moment and the two adjacent previous sampling points to the current moment to obtain the change rate of the process parameter; Based on the production situation of the same model camera module in the past, determine the dynamic weight value of the process parameter; Multiply the change rate of each process parameter by the dynamic weight value of the corresponding process parameter to obtain the weighted values of all process parameters; Perform a summation calculation on the weighted values of all process parameters to obtain the weighted summation value of the process parameters; Determine the adaptive coefficient according to the target module production line; Multiply the weighted summation value of the process parameters by the adaptive coefficient to obtain the evaluation value of the process parameter change; Obtain the current production duration and production standard cycle through the target module production line, and obtain the time difference of production based on the current production duration and production standard cycle; Use the exponential function to perform non-linear conversion on the time difference of production to generate the time sensitivity factor of production; Collect multi-spectral images and a standard optical feature library through the camera module, extract optical feature vectors in multiple dimensions from the multi-spectral images, and obtain standard optical feature vectors from the standard feature library; Based on the optical feature vectors and standard optical feature vectors, calculate the Euclidean distance value of the optics in the feature space; Multiply the Euclidean distance value of the optics by the time sensitivity factor of production to obtain the evaluation value of the optical feature difference; Linearly superimpose the evaluation value of the process parameter change and the evaluation value of the optical feature difference to obtain the comprehensive evaluation value of production; Determine the analysis threshold based on the production standard; Perform production change analysis on the current production module based on the comprehensive evaluation value of production and the analysis threshold to obtain a production change analysis report; Judge whether the defect detection scenario is satisfied based on the production change analysis report.

4. The method for detecting part defects based on computer vision according to claim 3, wherein When the defect detection scenario is satisfied, perform initial inspection shooting control, obtain an initial inspection shooting image, and perform substrate edge analysis on the initial inspection shooting image to determine whether there is an edge stability defect, which specifically includes the following steps: Generate an initial inspection shooting instruction when the defect detection scenario is satisfied; Send the initial inspection shooting instruction to the current production module, and perform initial inspection shooting through the current production module to obtain an initial inspection shooting image; Perform substrate edge analysis on the initial inspection shooting image, obtain substrate edge length data, obtain substrate material attribute parameters, the environmental temperature of the production line, and the process standard database; Compare the lengths of the corresponding edges of the substrate edge length data in combination with the substrate material attribute parameters, the environmental temperature of the production line, and the process standard database to determine whether there is an edge stability defect.

5. The method for detecting part defects based on computer vision according to claim 4, characterized in that, Compare the lengths of the corresponding edges of the substrate edge length data in combination with the substrate material attribute parameters, the environmental temperature of the production line, and the process standard database to determine whether there is an edge stability defect. The specific steps are as follows: Obtain the real-time working condition temperature value through the environmental temperature of the production line, and obtain the material expansion coefficient, material hardness value, and ductility value from the substrate attribute parameters; Retrieve the curvature stability threshold from the process standard database; Calculate based on the material expansion coefficient and the real-time working condition temperature value to obtain the dynamic length tolerance threshold; Calculate based on the material hardness value and the ductility value to obtain the length balance factor; Obtain the measured edge length and the standard theoretical length from the substrate edge length data, and obtain the absolute deviation value by comparing the measured edge length and the standard theoretical length of each edge; Perform normalization processing on the absolute deviation value using the dynamic length tolerance threshold to obtain the normalized absolute deviation value; Perform weight adjustment on the normalized absolute deviation value using the length balance factor to obtain the standardized length deviation value; Obtain the edge point coordinates from the substrate edge length data, and based on the edge point coordinates, use the five-point division method to obtain the curvature change rate; Perform discretization processing and maximum value processing on the curvature change rate in sequence to obtain the maximum mutation characteristic value, and perform normalization processing on the maximum mutation characteristic value using the curvature stability threshold to obtain the normalized mutation characteristic value; Perform reverse weight adjustment on the normalized mutation characteristic value using the length balance factor to obtain the scoring value of curvature stability; Add the standardized length deviation value and the scoring value of curvature stability to obtain the segmented comprehensive scoring value, and perform arithmetic averaging on the segmented comprehensive scoring value to obtain the average value of the comprehensive score; Determine the gradient classification for judging edge stability defects, and combine the average value of the comprehensive score to judge the substrate edge stability defects.

6. The method for detecting part defects based on computer vision according to claim 5, characterized in that, When there is an edge stability defect, perform edge defect alarm, which specifically includes the following steps: Generate an edge defect signal when there is an edge stability defect; Send the edge defect signal to multiple preset defect alarm addresses; Respond to the edge defect signal to perform edge defect alarm.

7. The method for detecting part defects based on computer vision according to claim 6, characterized in that, When there is no edge stability defect, the initial inspection captured image is subjected to substrate center point detection to determine whether there is a suspected center deviation, which specifically includes the following steps: When there is no edge stability defect, a center point detection instruction is generated; According to the center point detection instruction, the substrate center point of the initial inspection captured image is detected to obtain center point detection data; From the center point detection data, the module center point and the substrate center point are identified; According to a preset center point error range, the module center point is compared with the substrate center point, and the center point comparison result is recorded; According to the center point comparison result, it is judged whether there is a suspected center deviation.

8. The method for detecting part defects based on computer vision according to claim 7, wherein According to a preset center point error range, the module center point is compared with the substrate center point, and the center point comparison result is recorded. The specific steps are as follows: The three-dimensional coordinates of the module center point are determined by identifying the module center point; The three-dimensional coordinate components of the center point are extracted and generated from the three-dimensional coordinates of the module center point; Based on the material expansion coefficient and the real-time working condition temperature value, a dynamic tolerance threshold is obtained through calculation; The substrate deformation factor is obtained from the module production line; Based on the three-dimensional coordinate components of the center point and the dynamic tolerance threshold, a three-dimensional weighted deviation value is calculated, and the three-dimensional weighted deviation values are accumulated and summed to obtain the total three-dimensional weighted deviation value; The standard temperature value is obtained from the process standard database; Based on the real-time working condition temperature value and the standard temperature value, the absolute temperature difference is calculated; The absolute temperature difference is selected by an exponential function attenuation model to obtain a non-linear temperature compensation coefficient; The substrate deformation factor is multiplied by the non-linear temperature compensation coefficient to obtain a combined correction value; The total three-dimensional weighted deviation value is added to the combined correction value to obtain a composite deviation index; The deviation determination threshold is determined according to a preset center point error range; The composite deviation index is compared with the deviation determination threshold, and the comparison result is generated and recorded.

9. The method for detecting part defects based on computer vision according to claim 8, wherein When there is a suspected center deviation, a standard camera module is selected for contrast shooting control, a contrast shooting image is obtained, and defect contrast detection analysis is performed to output contrast detection information, which specifically includes the following steps: When there is a suspected center deviation, according to the current production module, a standard camera module is selected; A contrast shooting instruction is generated; The contrast shooting instruction is sent to the standard camera module, and contrast shooting is performed through the standard camera module to obtain a contrast shooting image. The current production module is detected by a light intensity sensor to obtain the current ambient light intensity value, and the surface roughness parameter of the part is obtained by a surface profiler; Combined with the current ambient light intensity value and the roughness parameter, the contrast shooting image is subjected to defect contrast detection analysis with the initial inspection captured image to output contrast detection information.

10. The method for detecting part defects based on computer vision according to claim 9, wherein, Combined with the current ambient light intensity value and the roughness parameter, the contrast shooting image is subjected to defect contrast detection analysis with the initial inspection captured image to output contrast detection information. The specific steps are as follows: A test image is obtained from the contrast shooting image, a standard image is obtained from the initial inspection captured image, and the roughness quantization value is determined by the roughness parameter; The test image and the standard image are processed using wavelet transform to generate a test feature map and a standard feature map; For each feature level in the test feature map and the standard feature map, the high-frequency detail component and the low-frequency contour component are extracted to obtain the processed feature map; The processed feature map is subjected to edge enhancement processing to obtain wavelet feature maps of each layer; Based on the current ambient light intensity value, a light intensity compensation mechanism is used to adjust the initial weight to obtain a dynamically adjusted weight parameter; Match the base noise threshold according to the roughness quantization value; The absolute value difference between the wavelet feature maps of each layer and the standard feature map is calculated layer by layer to obtain a feature difference value; The feature map difference value is divided by the base noise threshold of the corresponding level to obtain the feature difference value of each level; An exponential amplification process is applied to the feature difference values of each level to obtain the normalized and amplified feature difference values of each level; Gamma correction is used to process the test image and the standard image to generate the brightness histogram of the test image and the brightness histogram of the standard image; The probability distribution difference value is calculated through the brightness histogram of the test image and the brightness histogram of the standard image; The smoothing factor is used to process the zero-probability region in the probability distribution difference value to obtain the brightness distribution difference quantization value; The normalized and amplified feature difference values of each level are multiplied by the dynamically adjusted weight parameter to obtain the multi-scale feature fusion difference value; The brightness distribution difference quantization value is multiplied by the dynamically adjusted weight parameter to obtain the light robustness difference factor; The multi-scale feature fusion difference value and the light robustness difference factor are weighted and summed to obtain the comprehensive defect contrast index; The comprehensive defect contrast index is input into the defect classifier to obtain the defect classification result, and the comparison detection information is further confirmed based on the defect classification result.

11. A computer vision-based part defect detection system, characterized in that, The system applies the computer vision-based part defect detection method described in any one of claims 1 to 10 above. The system includes: A production monitoring and processing unit for performing production monitoring of the camera module, obtaining production monitoring data, analyzing the production monitoring data, and determining whether it meets the defect detection scenario; A substrate edge analysis unit for performing preliminary inspection shooting control when the defect detection scenario is met, obtaining a preliminary inspection shooting image, and performing substrate edge analysis on the preliminary inspection shooting image to determine whether there is an edge stability defect; An edge defect alarm unit for performing an edge defect alarm when there is an edge stability defect; A substrate center point detection unit for performing substrate center point detection on the preliminary inspection shooting image when there is no edge stability defect to determine whether there is a suspicious center deviation; A defect comparison detection unit for selecting a standard camera module when there is a suspicious center deviation, performing comparison shooting control, obtaining a comparison shooting image, and performing defect comparison detection analysis to output comparison detection information.

Images