Depth visual defect detection method and system
By comparing images of mechanical parts with standard texture images and pre-training models, the problem of misjudgment of atypical textures is solved, high-precision defect detection and risk assessment are achieved, and the reliability and efficiency of the detection system are improved.
Patent Information
- Application Number
- CN202510993748.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-18
- Publication Date
- 2025-10-17
AI Technical Summary
Existing technologies have difficulty accurately identifying atypical textures and real defects in mechanical manufacturing, resulting in a high false detection rate and an inability to meet large-scale production needs.
By obtaining part images and comparing them with standard texture images of the corresponding processing stage, atypical texture areas are extracted. Combined with pre-trained defect models and texture difference calculations, accurate defect positioning and risk assessment can be achieved.
It improves detection accuracy, reduces false detection rates, provides risk warnings for defect development areas, supports effective engineering processing suggestions, and meets the quality inspection needs of high-precision manufacturing scenarios.
Smart Images
Figure CN120807475A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of mechanical manufacturing quality detection, in particular to a deep visual defect detection method and a system based on the deep visual defect detection method. BACKGROUND
[0002] In the modern mechanical manufacturing field, the quality of the part surface is directly related to the performance stability, service life and safety of the whole machine. Especially in the application scenarios such as aviation, automobile and high-end equipment manufacturing which have high requirements on precision and reliability, small surface defects (such as cracks, peeling, pores and inclusions) may evolve into structural failure in the subsequent service process, resulting in serious consequences.
[0003] The traditional manual visual inspection method relies on experienced quality inspection personnel to observe the part surface by naked eye, which has the disadvantages of strong subjectivity, poor stability and low efficiency, and is difficult to be used continuously in large-scale and high-throughput production sites.
[0004] The existing image processing and recognition defect method performs well when facing typical texture background and defects with obvious contrast, but when facing complex texture background or non-typical surface morphology, it often has the problems of false detection or missed detection. In the mechanical processing process, due to tool wear, cooling liquid deposition or the influence of a specific process flow, the part surface may exhibit some non-typical but normal texture structures. The existing image processing and recognition defect method is easy to misjudge such non-typical texture as a defect, resulting in a high false positive rate and reducing the system reliability. SUMMARY
[0005] In view of the problem that the existing method cannot accurately identify the difference between non-typical texture and real defects, the present application provides a deep visual defect detection method.
[0006] In order to achieve the above purpose, the technical scheme of the present application is as follows:
[0007] In a first aspect, the present application discloses a deep visual defect detection method, comprising the following steps:
[0008] Obtaining a part image;
[0009] Comparing the part image with a standard texture image corresponding to the processing stage thereof, and then extracting a first region containing non-typical texture;
[0010] Calculating the similarity of the first region and a first data set one by one, and comparing the maximum similarity value with a preset first threshold value; wherein the first data set is a collection of part pictures with non-typical texture but no defects;
[0011] If less than the first threshold, the first region and the corresponding region of the reference picture are calculated for texture difference, the texture difference value is matched with different defect difference intervals, and the first defect existing in the first region is determined; wherein the reference picture is a standard part image of a processing stage of the part image;
[0012] determine whether the first defect is a core defect, if yes, rework or scrap processing;
[0013] Otherwise, the first defect is input to the pre-trained defect model to predict the defect development region;
[0014] Referring to the reference table representing the mapping relationship between the defect position, area and risk, the risk level corresponding to the defect development region is determined, and the corresponding suggestion is given.
[0015] In a second aspect, the application discloses a deep visual defect detection system, comprising a data acquisition module, a first comparison module, a second comparison module, a defect matching module, a data judgment module, a defect prediction module and a result output module.
[0016] The data acquisition module is used for acquiring a part image;
[0017] The first comparison module is used for comparing the part image with a standard texture image corresponding to its processing stage, and then extracting a first region containing atypical texture;
[0018] The second comparison module is used for calculating the similarity between the first region and the first data set one by one, and comparing the maximum similarity value with the first threshold; wherein the first data set is a set of part pictures without defects but with atypical texture;
[0019] The defect matching module is used for calculating the texture difference between the first region and the corresponding region of the reference picture if the first threshold is less than the first threshold, matching the texture difference value with different defect difference intervals, and determining the first defect existing in the first region; wherein the reference picture is a standard part image of a processing stage of the part image;
[0020] The data judgment module is used for judging whether the first defect is a core defect, if yes, rework or scrap processing;
[0021] The defect prediction module is used for inputting the first defect to the pre-trained defect model to predict the defect development region when the judgment result is no;
[0022] The result output module is used for determining the risk level corresponding to the defect development region by referring to the reference table representing the mapping relationship between the defect position, area and risk, and giving the corresponding suggestion.
[0023] In a third aspect, the present application discloses a computer terminal, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the deep visual defect detection method as described above when executing the program.
[0024] Compared with the prior art, the present application has the following beneficial effects:
[0025] 1. The present application compares the actual image with the standard texture image corresponding to the processing stage, extracts and focuses on the atypical texture area, realizes accurate positioning of the abnormal area, improves the detection accuracy, reduces the false detection rate through comparison of non-defect atypical texture data, adopts texture difference calculation and defect interval matching, and realizes fine-grained discrimination of defect types;
[0026] 2. The present application introduces a pre-trained defect model to predict the area where the defect may expand, supports early warning of defect risk, provides timeliness basis for maintenance measures such as repair, replacement or observation, and further provides corresponding engineering treatment suggestions. BRIEF DESCRIPTION OF DRAWINGS
[0027] The disclosed content of the present application is explained with reference to the accompanying drawings. It should be understood that the drawings are only for illustrative purposes, and are not intended to limit the scope of protection of the present application. In the drawings, the same reference numerals are used to refer to the same components. Among them:
[0028] Figure 1 a flowchart of the deep visual defect detection method introduced by the present application;
[0029] Figure 2 a flowchart of extracting the first area based on Figure 1 ;
[0030] Figure 3 a flowchart of obtaining the maximum similarity value based on Figure 1 ;
[0031] Figure 4 a flowchart of correcting the first area based on Figure 1 ;
[0032] Figure 5 a flowchart of verifying the first defect according to the curvature of the first area based on Figure 1 ;
[0033] Figure 6 a flowchart of checking the first defect again if the first area and the second area are still different after verification based on Figure 5 ;
[0034] Figure 7 a flowchart of calling the corresponding pre-trained defect model according to the part type based on Figure 1 ;
[0035] Figure 8 The block diagram of the deep visual defect detection system introduced in the present application is shown in the figure.
[0036] Figure 9 The structural schematic diagram of the computer terminal introduced in the present application is shown in the figure. DETAILED DESCRIPTION
[0037] It is easy to understand that, according to the technical scheme of the present application, a person skilled in the art can propose a plurality of structure modes and implementation modes which can be replaced with each other without changing the essential spirit of the present application. Therefore, the following specific embodiments and the accompanying drawings are only exemplary descriptions of the technical scheme of the present application, and should not be regarded as the whole or as the limitation or restriction of the technical scheme of the present application.
[0038] SUMMARY
[0039] In the prior art, the field of mechanical manufacturing has long relied on manual visual inspection and traditional image processing technology for part surface defect detection. Manual visual inspection has strong subjectivity and low efficiency, which is difficult to meet the needs of large-scale production. The traditional image processing method is easy to misjudge the normal non-typical texture as a defect under complex texture background, for example, the regular lines formed by tool wear or the temporary marks produced by cooling liquid deposition are often incorrectly identified as cracks or pores. Therefore, the prior art lacks the ability to differentiate the texture features dynamically changed in the processing stage, resulting in a high false positive rate.
[0040] In order to solve the above problems, it is found that the existing method does not consider the standard texture features in the processing stage, and cannot distinguish between normal process texture and real defects. After analysis, it is found that by establishing a standard texture library in the processing stage, the normal texture interference introduced by the process flow can be eliminated. Deep research shows that even if there is non-typical texture, it also needs to be verified again with historical defect-free samples to avoid misjudgment. Finally, a multi-level verification mechanism is formed: first filter normal process texture, then compare with the defect-free sample library, and finally accurately identify the defect type through difference interval matching.
[0041] After introducing the basic concept of the present application, the embodiments of the present application will be specifically introduced with reference to the accompanying drawings.
[0042] Exemplary system
[0043] As shown in Figure 1 The deep visual defect detection method is introduced, including the following steps:
[0044] S100. Acquire part image.
[0045] S200. Compare the part image with the standard texture image of its corresponding processing stage to extract a first region containing atypical texture; wherein the standard texture image is a reference image of the normal part surface texture of the same processing stage, and is generated by feature fusion of multiple batches of qualified part images.
[0046] S300. Calculate the similarity of the first region and the first data set one by one, and compare the maximum similarity value with the preset first threshold value; wherein the first data set is a collection of part pictures with atypical texture but no defects.
[0047] S400. If less than the first threshold value, calculate the texture difference of the first region and the corresponding region of the reference picture, match the texture difference value with different defect difference intervals, and determine the first defect existing in the first region; wherein the reference picture is a standard part image of the processing stage where the part image is located.
[0048] S500. Determine whether the first defect is a core defect, if yes, then rework or scrap; wherein the core defect includes a predefined irreparable defect type and / or a defect position in a stress concentration area of the part.
[0049] S600. Otherwise, input the first defect into the pre-trained defect model to predict the defect development area.
[0050] S700. Determine the risk level corresponding to the defect development area by referring to the reference table representing the mapping relationship between the defect position, area and risk, and give the corresponding suggestion.
[0051] The embodiment effectively distinguishes between normal process texture and real defects, avoiding misjudgment of atypical textures such as tool wear or cooling liquid residue as defects. By dynamically matching the defect difference interval, the recognition accuracy of subtle defects such as cracks and pores is improved. Combined with the defect expansion prediction function, potential risk areas can be warned in advance, providing quantitative basis for production decision. Finally, while ensuring detection accuracy, unnecessary part scrap loss is reduced.
[0052] The overall scheme of the deep visual defect detection method is introduced above, and S200. Compare the part image with the standard texture image of its corresponding processing stage to extract a first region containing atypical texture is described in detail as follows. Figure 2
[0053] S201. Preprocess the part image, and extract first texture features from the processed part image;
[0054] S202. Calculate the residual by subtracting the second texture features corresponding to the position of the standard texture image from the first texture features;
[0055] S203. Determine whether the residual is greater than the preset residual threshold. If yes, select it as the first region.
[0056] After obtaining the part image, in order to facilitate comparison with the standard texture image, the part image needs to be processed accordingly. Specifically, one or more of the following preprocessing operations are performed on the part image: grayscale, Gaussian blur or median filter, adaptive histogram equalization, edge enhancement, etc. Remove noise and unify image style.
[0057] Then, through feature point matching, focus on the key surface texture area, and avoid background interference. After feature point matching, affine transformation or perspective transformation is performed to ensure one-to-one correspondence between the current image and the standard texture image in space. Then, normalization is performed, and finally feature extraction and fusion are performed to convert the current image into a comparable texture feature space. Feature extraction can use CNN to extract convolutional layer texture features, or Gabor filter extraction, or use pre-trained networks such as ResNet and EfficientNet to extract feature maps.
[0058] Define the first texture feature extracted as , and the second texture feature as , and calculate the residual: ; wherein , represents the i-th dimensional feature component; represents the feature vector length of each pixel point, which depends on the type of texture feature.
[0059] Set the residual threshold as:
[0060] By judging whether each pixel position is abnormal, the atypical region mask (first region) is obtained: .
[0061] Finally, the first region is: .
[0062] It should be emphasized that in order to accurately obtain the standard texture image corresponding to the part image of the processing stage, the part image can be taken by each processing stage fixed image acquisition node on the production line, and each node is automatically labeled with the processing stage. If the image is not labeled with the stage, an image classifier (such as ResNet, MobileNet) can be introduced to determine the processing stage. After inputting the part image, the processing stage label is output.
[0063] The application eliminates environmental interference through pretreatment, adopts a combination of feature space difference calculation and threshold judgment, and accurately distinguishes normal machining texture from real surface defects. The method can effectively avoid misjudging non-typical but qualified textures such as tool tracks and cooling liquid residues as defects, and improves the reliability of the detection system.
[0064] The step S200 is specifically introduced above, and the step S300 of performing similarity calculation on the first region and the first data set one by one and taking the maximum similarity value and the preset first threshold value is specifically described as follows: Figure 3
[0065] S301. Extract the texture features of each picture in the first data set as third texture features;
[0066] S302. Perform cosine similarity calculation on the first texture features of the first region and the third texture features to obtain a similarity value;
[0067] S303. Arrange the plurality of similarity values in descending order, and take Top1 as the maximum similarity value.
[0068] Suppose that the first data set has N images, each of which is a non-typical texture but has no defects. For each sample image j, the cosine similarity between the image and the first region is calculated as follows: ; wherein, denotes the texture feature vector of the first region, denotes the Euclidean length.
[0069] Take the maximum similarity value (Top1) as follows: .
[0070] If the first threshold value (based on the adaptive adjustment of historical sample distribution) is met, the subsequent steps are continued.
[0071] The application solves the misjudgment problem of non-typical texture and real defect, and reduces the false detection rate caused by texture similarity. Through the comparison of the reference feature library and the similarity sorting mechanism, it is ensured that the detection system only starts the defect analysis process for unknown texture mode, reduces the consumption of invalid calculation resources, and improves the detection efficiency and result reliability.
[0072] The step S300 is specifically introduced above, and before the step of performing similarity calculation on the first region and the first data set one by one, the first region is modified as follows: Figure 4
[0073] S311. Obtain luminance data of the first region and obtain light intensity data of an environment in which the part image is taken; wherein the luminance data is a pixel gray value collected by an image sensor, and the light intensity data represents an influence of an external light source on imaging;
[0074] S312. If the luminance data of the first region is greater than a preset third threshold value, mark the first region as a region to be corrected;
[0075] S313. Calculate a compensation coefficient by comparing preset reference data with the light intensity data;
[0076] S314. Retrieve a reflectivity model of a processing stage in which the part image is located in a pre-stored reflectivity model database, and adjust the reflectivity model according to the compensation coefficient;
[0077] S315. Input the region to be corrected into the adjusted reflectivity model to obtain a corrected first region.
[0078] That is, first, luminance data of the first region and light intensity data of an environment in which the part image is taken obtained from a sensor or an environmental light illuminometer :
[0079] Region average luminance ;
[0080] wherein, represents a gray value of the image at a pixel .
[0081] If the region average luminance is greater than a preset third threshold value (the third threshold value), mark the first region as a region to be corrected.
[0082] Obtain a standard light value used in an ideal or calibration state as reference data, that is, a reference shooting light intensity , and calculate a compensation coefficient .
[0083] According to a processing stage label, retrieve a corresponding reflectivity model in an existing reflectivity model database , The parameter in the reflectivity model is affected by light, and needs to be corrected by using the compensation coefficient, that is, , is an empirical adjustment coefficient, used to control the compensation degree.
[0084] Use the adjusted model to perform reflectivity back calculation on the original region image:
[0085] The corrected gray value : .
[0086] Finally, a corrected first region image is obtained and re-extract the texture features thereof:
[0087] .
[0088] The application can reduce the misjudgment of atypical texture caused by environmental light fluctuation or difference in surface reflectivity of materials. For example, in the case of strong reflection of oil film on the surface of a machined part, the corrected reflectivity model can accurately identify the reflection area as normal texture, avoiding misjudgment as a crack or peeling defect. At the same time, the method adapts to changes in light in different processing environments through a dynamic compensation mechanism, improving the robustness of the defect detection system in complex industrial scenarios.
[0089] The above specifically introduces the process of correcting the first region after determining the first region, and the following describes step S400 in detail. If it is less than the first threshold, the texture difference of the first region and the corresponding region of the reference picture is calculated, the texture difference value is matched with different defect difference intervals, and the first defect existing in the first region is determined. The specific steps are as follows:
[0090] If the texture difference is less than the first threshold (the first threshold), the texture difference ; wherein, represents the texture feature vector of the first region, represents the texture feature vector of the corresponding region of the reference picture.
[0091] The corresponding defect type is found through the texture difference. For details, refer to the following table:
[0092] Table 1: Defect type and texture difference interval comparison table
[0093] Defect type Difference interval D Surface contamination [CD]D1≤ D < D2 Crack [D2≤ D < D3] Material deficiency D3 < D < D4 Bulge [D4≤D<D5] ... ...
[0094] By calculating the texture difference, subtle texture changes can be captured, and detection accuracy can be improved. Only when the texture difference is less than the first threshold, the texture analysis is triggered, avoiding redundant calculation of full image scanning and reducing processing load. The defect difference interval can be flexibly adjusted according to specific applications, making the scheme easy to extend to different scenarios and improving universality.
[0095] The above specifically introduces step S400, and the following describes step S500 in detail. If the first defect is a core defect, rework or scrap processing is performed. The specific steps are as follows:
[0096] First, the core defect includes predefined non-repairable defect types and / or defect positions in the stress concentration area of the part.
[0097] Define the defect type set as: .
[0098] Wherein, the set of unrepairable defect types is: .
[0099] The type judgment formula is: ; wherein, represents the first defect type.
[0100] The definition of the defect position is , and the stress concentration area is ⊆ part surface area
[0101] The position judgment formula is: .
[0102] If any condition (type or position) is met, it is considered a core defect: ;
[0103] If 1, rework or scrap is required.
[0104] The above specifically introduces step S500, and the following will be described in detail before judging whether the first defect is a core defect, including verifying the first defect according to the curvature of the first area, as shown in Figure 5 , the specific steps are as follows:
[0105] S501. Obtain other view images of the part image at the same time stamp, match the images at different angles, and construct a three-dimensional model of the part; wherein, the other view images are synchronously captured by a multi-view camera;
[0106] S502. Calculate the curvature of the part image using differential geometry method;
[0107] S503. Calculate the curvature difference of each point of the part image and the average curvature of the adjacent area, and judge whether the calculation result is greater than the preset second threshold value. Yes, mark as a candidate area;
[0108] S504. Judge whether there is a first area in the candidate area. Yes, extract the curvature of the first area and match the corresponding defect;
[0109] S505. If the matched defect is not the same as the first defect, obtain the next image of the part image for verification;
[0110] S506. If the same, perform subsequent operations.
[0111] That is, obtain a set of images captured by a multi-view camera .
[0112] Construct a point cloud model using parallax method or structured light, etc. .
[0113] Use differential geometry method to calculate the curvature of the three-dimensional model each surface point of the 3D model On the other hand, the principal curvatures , : Gaussian curvature , mean curvature .
[0114] For each point p, compare its mean curvature difference with the surrounding neighborhood:
[0115] Curvature jump value .
[0116] Determine whether it is greater than the second threshold: . Curvature jump marker function
[0117] Construct a candidate curvature anomaly region set: .
[0118] Let the corresponding projection of the first region in the three-dimensional model be Determine whether it belongs to the candidate region:
[0119] ⇒ Curvature verification is performed.
[0120] Then extract its feature curvature vector Match the corresponding defect data, which can be seen in Table 2:
[0121] Matched defect type ; wherein, Indicates a defect sample library with labeled curvature features.
[0122] Let the matching result be The previous stage detection result (first defect) is If , call the next moment image and reconstruct the three-dimensional model Repeat the above curvature judgment process for verification.
[0123] If the defect is consistent, it is determined to be true, and the next stage is entered to determine whether it is a core defect.
[0124] Table 2: Defect type and curvature feature comparison table
[0125] Defect type Gaussian curvature K K Mean curvature H H Principal curvatures #timg#, #timg# features Geometric property description Dimple K>0 H<0 negative value Both principal curvatures are negative and close in absolute value Concave region, surface retraction Convex hull K>0 H > 0, positive value Both principal curvatures are positive Local bulge Scratch K≈0 or K<0 H ≠ 0, less stable One principal curvature is significant and the other is close to 0 Long strip structure, curvature asymmetry Crack K0, minimum or negative infinity H fluctuates sharply One principal curvature is abrupt and the other is usually negative Structural fragmentation, principal curvature direction mutation is large Sharp angle / edge burr K<0 H is relatively high (local protrusion) Curvature gradient is large and normal changes sharply Irregular edge, surface burr Porosity / inclusion K > 0 or K ≈ 0 H<0 or fluctuation Central point negative curvature, edge curvature changes steeply Point-like depression or structural anomaly
[0126] The application can effectively distinguish real defects from atypical surface geometric features, and reduce the false detection rate in a complex texture background. Through the dual verification mechanism of geometric features and texture features, the recognition accuracy of subtle physical defects is improved. Temporary interference factors are excluded by dynamic time verification, enhancing the anti-interference ability of the detection system. Finally, the accuracy of defect judgment is significantly improved, meeting the quality detection needs of high-precision manufacturing scenes.
[0127] The process of verifying the first defect based on the curvature of the first region is specifically introduced above. If the defect based on curvature matching is still different from the first defect after verifying the image obtained at the next moment of the part image, the first defect is checked, which is described in detail as follows: Figure 6
[0128] S511. Mark the first region corresponding to the first defect that does not match after verification as a suspicious region;
[0129] S512. Obtain the spectrum data of the suspicious region, check it according to the mapping relationship between the spectrum and the defect, and make the following decisions according to the checking result:
[0130] (1) If the suspicious region is indeed the first defect, save the image to the database, and do not perform curvature verification processing in subsequent detection if there is the same image;
[0131] (2) If the suspicious region is indeed the curvature matching defect, save the image to the database, and use the curvature verification result as a reference in subsequent detection if there is the same image;
[0132] (3) If the suspicious region is other, save the image to the database as sample data for subsequent parameter correction.
[0133] Among them, the spectrum data is collected by a near-infrared spectrometer or a hyperspectral imaging device.
[0134] If , and the image at the next moment is still inconsistent, mark the first region as a suspicious region, that is: .
[0135] For all pixel points p in the suspicious region, obtain the spectral reflectance curve: .
[0136] Calculate the overall spectral feature vector of the region: .
[0137] The spectrum-defect mapping model is trained by near-infrared, short-wave infrared or visible light multi-band images , and the defect is identified by the spectrum model: .
[0138] If : (Next skip curvature verification).
[0139] If , (Next curvature as reference).
[0140] If , (As a model correction sample).
[0141] The application solves the misjudgment problem caused by the inconsistency between the curvature verification and the texture difference judgment result, improves the defect classification accuracy by using the physical characteristics of the spectrum data. The dynamic marking mechanism of the suspicious area reduces the invalid review operation, and the database storage rule optimizes the processing efficiency of the subsequent detection process. The continuous accumulation of sample data provides data support for the iterative upgrade of the defect model, and forms a virtuous cycle of self-improvement of detection accuracy.
[0142] The process of checking the first defect is specifically introduced above, and the step S600. Otherwise, the first defect is input to the pre-trained defect model to predict the defect development area. The specific steps are as follows:
[0143] Extract the multi-dimensional features of the current defect area as the model input, and the feature vector of the first area is :
[0144] ; wherein, represents the current image, represents the first defect type. Wherein, the features include local texture statistics, geometric shape, defect type code, defect center position coordinates, defect history change information, etc.
[0145] The feature vector is input into the defect development model, and the output is a predicted region mask: ; wherein, represents the defect model for predicting defect development, and the specific training process is as follows:
[0146] Obtain image sequences of multiple batches of parts collected at different processing stages, corresponding defect annotation masks, mark defect initial regions and development regions, defect type labels such as cracks, pits, and material foreign matters. Organize them into training samples, and each training sample is composed of part image , defect initial mask , defect type label and defect development region at several time points.
[0147] Define the model input: ; wherein, The embedding vector of the defect type.
[0148] Using the Mask2Former network structure, a model is constructed, that is .
[0149] The loss function is set as ,
[0150] ; wherein, represents the predicted value of pixel point i, represents the true label of pixel point i, represents the total pixel intensity of the predicted defect area, represents the total number of pixels in the actual defect area.
[0151] The comprehensive loss function is obtained as . , represents the weight coefficient.
[0152] Then other configurations are set, such as using AdamW optimizer, setting learning rate, etc. After the configuration is completed, the model is trained. Since the texture, position, defect type and other multi-modal information are fused during the training process, the defect expansion probability graph for the part is constructed, and the multi-loss joint optimization strategy is adopted, so the integrity and accuracy of the predicted area are improved.
[0153] The step S600 is specifically introduced above, and before the first defect is input to the pre-trained defect model to predict the defect development area, the corresponding pre-trained defect model for prediction is called according to the part type, which is specifically described as shown in Figure 7 The specific steps are as follows:
[0154] S601. Extract the class ID of the part image;
[0155] S602. Map the class ID to the defect model database to find the corresponding pre-trained defect model.
[0156] Among them, the class ID refers to a unique code used to identify the specific type of the part, which can be realized by using a convolutional neural network classifier based on part image feature extraction. Through the feature extraction and classification operation of the trained classifier on the input image, the corresponding part type code is output. This feature can accurately identify the type attribute of the measured object, avoiding the feature confusion problem when different types of parts share the detection model.
[0157] The mapping function is ; wherein, represents the class identifier, represents the defect prediction model of the corresponding part type.
[0158] The application can automatically match the prediction model matched with different part types, solve the defect expansion prediction misalignment problem caused by part structure or material difference. For example, in the detection of automobile engine cylinder body, the system can accurately call the cylinder special model to predict the expansion trend of blowhole defects in high temperature environment, avoiding the general model misjudging the blowhole as stable form and ignoring its potential risk. This type of modeling mechanism improves the reliability and applicability of defect development area prediction.
[0159] The above specifically introduces the prediction according to the part type to call the corresponding pre-trained defect model, and the following S700. The reference table for determining the risk level corresponding to the defect development area according to the reference table for representing the defect position, area and risk mapping relationship is given, and the corresponding suggestion is described in detail. The reference table is shown in the following table 3:
[0160] Table 3: Defect position, area and risk reference table
[0161] Defect location Area range (mm²) Risk level Suggested treatment measures Stress concentration area A>3.0 High risk Suggest scrapping or stopping processing 1.0<A≤3.0 Medium risk Rework or decision after non-destructive testing A≤1.0 Low risk Record archiving, continue processing Near the weld A>4.0 High risk Scrap or replace the welded structure 2.0<A≤4.02 Medium risk Weld re-inspection or repair A≤2.0 Low risk Acceptable, suggest regular review Edge area A>5.0 High risk Suggest rework, if necessary, return to factory for processing 2.0<A≤5.0 Medium risk Marked for processing, monitor crack development A≤2.0 Low risk Allow use, no impact on function or safety Non-critical area A>10.0 Medium risk Determine whether to repair after process evaluation A≤10.0 Low risk Directly put into subsequent process
[0162] In order to facilitate the understanding of the above embodiment, the following will take one specific application scene of the above embodiment as an example for description:
[0163] Through the multi-view industrial camera array, the blade is imaged at multiple angles after the spraying process is completed, and the blade image I under the current process is obtained.
[0164] The image I is compared with the standard texture image I of the spraying stage S The feature comparison is performed using Gabor filter or texture CNN to extract 、
[0165] Calculate the residual error , locate the area with residual error value exceeding the residual error threshold , and mark it as the first area .
[0166] The first area is compared with the texture features of the samples in the first data set (image set with normal texture but process deviation) .
[0167] The first threshold value), the texture difference degree of and the corresponding area of the reference picture is analyzed, and the texture difference degree is obtained. is mapped to the defect difference interval table, and judged as the first defect (such as surface peeling).
[0168] Check if the defect appears in the stress concentration area such as the blade root or the blade tip, or if it is an unrepairable material peeling; if so, immediately scrap or rework.
[0169] If not, reconstruct the 3D model of the blade using multi-view images, calculate the Gaussian curvature and mean curvature of the first region using differential geometry method, and if the curvature changes significantly, mark it as a curvature mutation area. Further verify if the defect and the curvature anomaly are consistent, if not, get the next frame of image for verification to prevent false positives.
[0170] The verification process is to perform hyperspectral imaging on the first region, extract the spectral features, and use the spectral-defect mapping model to further verify whether it is a surface physical damage, chemical corrosion or foreign matter residue. According to the verification result, adjust the subsequent processing strategy, and put the image sample into the database.
[0171] If it is confirmed as a non-core defect, input it into the defect prediction model corresponding to the category of the blade (mapped through the category ID), and get the future development trend chart, including area expansion and risk distribution.
[0172] According to the development area position and area, refer to the pre-defined risk mapping table, and output the corresponding suggestions, such as surface layer depth > 20 μm and located at the root = high risk, output high risk scrap suggestion.
[0173] In summary, the method effectively distinguishes between normal texture and real defects through multi-stage texture comparison, similarity threshold judgment and defect development prediction, and solves the problem of high false positive rate and inability to evaluate defect risk in traditional detection methods, with the advantages of improving detection accuracy and predicting risk level.
[0174] Exemplary system
[0175] The embodiment introduces a deep visual defect detection system, which includes a data acquisition module, a first comparison module, a second comparison module, a defect matching module, a data judgment module, a defect prediction module and a result output module.
[0176] The data acquisition module is used to acquire part images;
[0177] The first comparison module is used to compare the part images with the standard texture images corresponding to the processing stage, and then extract the first region containing atypical texture;
[0178] The second comparison module is used to calculate the similarity of the first region and the first data set one by one, and compare the maximum similarity value with the first threshold value; wherein the first data set is a collection of part pictures with atypical texture but no flaws;
[0179] The defect matching module is configured to, if the first region is smaller than the first threshold value, calculate a texture difference of the first region and a corresponding region of a reference image, match the texture difference value with different defect difference intervals, and determine a first defect existing in the first region; wherein the reference image is a standard part image of a processing stage in which the part image is located.
[0180] The data judgment module is configured to judge whether the first defect is a core defect, and if yes, perform rework or scrap processing.
[0181] The defect prediction module is configured to, if the determination result is no, input the first defect into a pre-trained defect model to predict a defect development region.
[0182] The result output module is configured to determine a risk level corresponding to the defect development region by referring to a reference table representing a mapping relationship between a defect position, an area, and a risk, and give a corresponding suggestion.
[0183] The system effectively distinguishes normal texture variation from real defects in the mechanical processing process, reduces the false detection rate in a complex texture background, realizes accurate identification of defect types and quantitative evaluation of risk levels, improves the reliability of the detection system, shortens the average detection time of a single part through an automatic defect classification and disposal process, and meets the high-throughput detection demand of an industrial site.
[0184] An exemplary computer terminal
[0185] As shown in Figure 9 The embodiment introduces a computer terminal, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and is characterized in that the processor implements the steps of the deep visual defect detection method as described above when executing the program.
[0186] The processor can be one or multiple, Figure 9 In this embodiment, the processor and the memory are connected through a bus or other means, wherein Figure 9 In this embodiment, the processor and the memory are connected through a bus or other means, wherein
[0187] The deep visual defect detection method can be applied in the form of software, such as a program designed to run independently, installed on a computer terminal, which can be a computer, a smart phone, etc. It can also be designed as an embedded program running on a computer terminal, such as a single-chip microcomputer.
[0188] The operating system, the network communication module, the data processing module and the application program for implementing the deep visual defect detection method are stored in the memory. The operating system can be Windows, Linux, Android or iOS, etc. The network communication module is used for communicating with data sources such as data acquisition devices to obtain real-time data. The data processing module is used for pre-processing, format conversion and quality control of the obtained data.
[0189] The application program includes data acquisition module, first comparison module, second comparison module, defect matching module, data judgment module, defect prediction module and result output module, etc. functional modules, which correspond to each step in the exemplary method.
[0190] The technical scope of the present application is not limited to the above description, and those skilled in the art can make various modifications and modifications to the above embodiments without departing from the technical idea of the present application, and these modifications and modifications should be within the protection scope of the present application.
Claims
1. A method for detecting defects by deep vision, characterized in that: It includes the following steps: Acquire part images; Comparing the part image with a standard texture image corresponding to the processing stage, thereby extracting a first region containing an atypical texture; Calculating similarity between the first region and a first data set one by one, and comparing the maximum similarity value with a preset first threshold; wherein the first data set is a collection of images of parts with atypical textures but no defects; If the value is less than a first threshold, calculating the texture difference between the first region and the corresponding region of a reference image, matching the texture difference value with the difference intervals of different defects, and determining the first defect in the first region; wherein the reference image is a standard part image of the part image at the processing stage; Determine whether the first defect is a core defect, and if so, rework or scrap the product; Otherwise, the first defect is input into a pre-trained defect model to predict the defect development area; Refer to the preset reference table that characterizes the relationship between defect location, area and risk mapping to determine the risk level corresponding to the defect development area and give corresponding suggestions.
2. The method for detecting depth defects according to claim 1, wherein: The specific steps of comparing the part image with the standard texture image of the corresponding processing stage and then extracting the first region containing the atypical texture are as follows: preprocessing the part image and extracting a first texture feature from the processed part image; Calculate the difference between the first texture feature and the second texture feature corresponding to the position of the pre-extracted standard texture image to obtain a residual; It is determined whether the residual is greater than a preset residual threshold, and if so, the region is selected as the first region.
3. The depth vision defect detection method according to claim 1, characterized in that: The specific steps of calculating the similarity between the first region and the first data set one by one are as follows: Extracting texture features of each image in the first data set as third texture features; Performing cosine similarity calculation on the first texture feature of the first region and the third texture feature to obtain a similarity value; Arrange multiple similarity values in descending order and take the top 1 as the maximum similarity value.
4. The depth vision defect detection method according to claim 1, characterized in that: Before calculating the similarity between the first region and the first data set one by one, the first region is also corrected. The specific steps are as follows: Acquiring brightness data of the first area and acquiring light intensity data of the environment in which the part image is captured; If the brightness data of the first area is greater than a preset third threshold, marking the first area as an area to be corrected; Calculating the ratio of the preset reference data to the light intensity data to obtain a compensation coefficient; Retrieving the reflectivity model of the processing stage of the part image from a pre-stored reflectivity model database, and adjusting the reflectivity model according to the compensation coefficient; The area to be corrected is input into the adjusted reflectivity model to obtain a corrected first area.
5. The depth vision defect detection method according to claim 1, characterized in that: Before determining whether the first defect is a core defect, the method further includes verifying the first defect according to the curvature of the first area. The specific steps are as follows: Acquire images of the part from other perspectives at the same timestamp as the image of the part, match the images from different angles, and construct a three-dimensional model of the part; calculating the curvature of the part image using differential geometry methods; Calculate the difference between the curvature of each point in the part image and the average curvature of the adjacent area, and determine whether the calculated result is greater than a preset second threshold. If so, mark it as a candidate area; Determine whether there is a first region in the candidate region, and if so, extract the curvature of the first region and match the corresponding defect; If the matched defect is different from the first defect, obtaining an image of the part at the next moment for verification; If they are the same, perform the subsequent operation.
6. The depth vision defect detection method according to claim 5, characterized in that: After obtaining and verifying the image of the part at the next moment, if the defect based on curvature matching is still different from the first defect, the first defect is checked, and the specific steps are as follows: Marking a first area corresponding to the first defect that still does not match after verification as a suspicious area; Obtain the spectrum data of the suspicious area, verify it with reference to the mapping relationship between the spectrum and the defect, and make the following decisions based on the verification results: (1) If the suspicious area is indeed the first defect, the image is saved in the database, and the curvature verification process is not performed when the same image is found in subsequent inspections; (2) If the suspicious area is indeed a defect that matches the curvature, the image will be saved in the database. If the same image exists later, the curvature verification result will be used as the basis; (3) If the suspicious area is found to be other, the image is saved in the database as sample data for subsequent correction parameters.
7. The depth vision defect detection method according to claim 1, characterized in that: Otherwise, before inputting the first defect into the pre-trained defect model to predict the defect development area, the method further includes calling the corresponding pre-trained defect model for prediction according to the part type, and the specific steps are as follows: Extracting a category ID from the part image; The category ID is mapped to the defect model database to find the corresponding pre-trained defect model.
8. The depth vision defect detection method according to claim 1, characterized in that: The core defects include predefined irreparable defect types and / or the defect locations are located in stress concentration areas of the parts.
9. A deep vision defect detection system, characterized in that: It includes: A data acquisition module, which is used to acquire part images; a first comparison module, configured to compare the part image with a standard texture image corresponding to the processing stage, thereby extracting a first region containing an atypical texture; a second comparison module configured to calculate similarities between the first region and a first data set one by one, and compare the maximum similarity value with a preset first threshold; wherein the first data set is a collection of images of parts with atypical textures but no defects; a defect matching module configured to calculate a texture difference between the first region and a corresponding region of a reference image if the texture difference is less than a first threshold, match the texture difference value with different defect difference intervals, and determine a first defect in the first region; wherein the reference image is a standard part image of the part image at the processing stage in which the part image is located; A data judgment module, which is used to judge whether the first defect is a core defect, and if so, to rework or scrap the product; A defect prediction module, configured to input the first defect into a pre-trained defect model to predict a defect development area when the determination result is negative; The result output module is used to determine the risk level corresponding to the defect development area by referring to a preset reference table that characterizes the relationship between defect location, area and risk mapping, and to provide corresponding suggestions.
10. A computer terminal, characterized in that It includes a memory, a processor, and a computer program stored in the memory and runnable on the processor, and is characterized in that when the processor executes the program, the steps of the depth vision defect detection method according to any one of claims 1 to 8 are implemented.