Defect detection method, device and system for large special-shaped metal workpiece

Abstract

The invention relates to a defect detection method, device and system for a large special-shaped metal workpiece. The defect detection method for the large special-shaped metal workpiece comprises the following steps: acquiring a multi-camera shooting image of the special-shaped workpiece passing through a semi-closed light field assembly, preprocessing the multi-camera shooting image, and inputting a plurality of preprocessed special-shaped workpiece images into a rough defect detection model for defect detection, obtaining a plurality of preliminary detection results; then, judging whether all the preliminary detection results show that defects do not exist or not: if not, considering that defects exist in the detection position of the current special-shaped workpiece, and completing defect detection; and if yes, further inputting the plurality of preprocessed special-shaped workpiece images into a fine defect detection model for defect detection to obtain a final detection result, and completing defect detection. According to the defect detection method for the large special-shaped metal workpiece, the imaging quality of detail areas of all parts of the special-shaped workpiece is remarkably improved, and the omission ratio is effectively reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual inspection, in particular to a defect detection method and device for large special-shaped metal workpieces, a defect detection system for a stamping production line of automobile special-shaped workpieces, an electronic device and a computer storage medium. BACKGROUND

[0002] Die stamping is one of the four major processes in an automobile workshop, mainly using multiple mechanical die stamping methods to form steel and aluminum materials, thereby obtaining vehicle body parts required by automobile design, and the vehicle body parts are generally special-shaped and large-sized. At the same time, during stamping, the vehicle body parts are prone to be affected by factors such as wear of some parts of the stamping die, foreign matter, stamping temperature, etc., resulting in some minor defects of the parts after one-time stamping forming. If the defects such as cracks, scratches, and pits of the parts after stamping forming are not discovered in time, the mechanical properties and service safety of the parts will be seriously affected when they enter the next process. Therefore, the safety requirement of automobile body part manufacturing is extremely high, and the corresponding detection system needs to have extremely low omission rate and defect detection performance.

[0003] In the traditional technology, some production lines still use manual visual inspection method, i.e., multiple inspectors gradually check the parts on the conveying line. However, although manual detection has high flexibility, it requires a large number of human resources and long-time operation, which is prone to cause human fatigue, thereby causing the omission rate to rise, and the detection results are difficult to quantify and save, which is not conducive to subsequent tracing and analysis of the detection results.

[0004] Therefore, the existing technology usually uses machine vision detection methods such as artificial neural network to replace manual detection, such as YOLO series (You Only Look Once), Faster R-CNN, etc. The specific method is to collect the surface image of the part by the camera, extract multi-scale features through the convolution layer in the model, and perform feature fusion and detection on the extracted multi-scale features to obtain the category and corresponding position of the defect, so as to realize automatic defect detection.

[0005] However, when the existing technology is used for large special-shaped workpieces, due to the significant difference in curvature and inclination angle of each part of the special-shaped workpiece, the position details of each large special-shaped workpiece reflect light, affecting the image quality, thereby affecting the information loss of the model in the feature extraction stage, and further causing the defect accuracy to decrease, which is prone to cause omission and other misjudgment phenomena. On the other hand, macro defects (such as obvious cracks and notches) of large special-shaped workpieces are easy to be detected, but subtle defects (such as slight scratches and shallow cracks) are easy to be submerged in background noise under the existing imaging conditions, resulting in a decrease in detection accuracy.

[0006] Therefore, the prior art has the problems of low detail capturing ability of strong light reflection surface, weak defect detection performance and high missing detection rate in high safety detection of large special-shaped metal workpieces. SUMMARY

[0007] Therefore, the purpose of the present application is to provide a defect detection method for large special-shaped metal workpieces.

[0008] A defect detection method for large special-shaped metal workpieces, comprising the following steps:

[0009] S1: obtaining multi-position shooting images of the special-shaped workpiece passing through the semi-closed light field assembly;

[0010] The multi-position shooting images include positive angle high exposure images, positive angle low exposure images, negative angle high exposure images and negative angle low exposure images.

[0011] S2: preprocessing the multi-position shooting images of the special-shaped workpiece passing through the semi-closed light field assembly to obtain a plurality of preprocessed special-shaped workpiece images;

[0012] S3: performing rough defect detection on the plurality of preprocessed special-shaped workpiece images to obtain a plurality of preliminary detection results;

[0013] S4: judging whether all preliminary detection results are defect-free: if not, it is considered that the current position of the special-shaped workpiece has defects, and the defect recognition is completed; if yes, step S5 is executed;

[0014] S5: performing fine defect detection on the plurality of preprocessed special-shaped workpiece images to obtain a final detection result.

[0015] The defect detection method for large special-shaped metal workpieces of the present application, compared with the prior art, by obtaining high exposure images and low exposure images of positive and negative angles under multi-position setting, significantly enhances the imaging quality of detail areas of each part of the special-shaped workpiece, especially in the case of strong light reflection on the workpiece surface, large curvature change or complex inclination angle, the complementary imaging of high and low exposure can effectively reduce the missing of detail information caused by reflection or shadow in the feature extraction stage, thereby providing more abundant and stable feature information basis for subsequent detection.

[0016] Secondly, by introducing the two-stage detection strategy of rough defect detection and fine defect detection in the detection process, in the application scene with high safety level, the missing detection rate can be reduced to the maximum extent with the most strict judgment strategy, thereby improving the reliability and consistency of the detection result.

[0017] Further, the rough defect detection is performed by a rough defect detection model, and the rough defect detection model comprises a backbone network, a neck network and a detection head.

[0018] The backbone network is used for multi-scale feature extraction on any preprocessed special-shaped workpiece image, and outputs a plurality of feature maps of different scales;

[0019] The neck network is used for feature fusion on the plurality of feature maps of different scales, and outputs a plurality of fused feature maps;

[0020] The detection head is used for classification and regression on the fused feature maps of different scales, and outputs defect categories, position box coordinates and confidence, to form a preliminary detection result of the current special-shaped workpiece image;

[0021] The step S4 further includes the following sub-steps:

[0022] If there is a position box of a defect category in the preliminary detection result of any special-shaped workpiece image, and the confidence of the position box is greater than or equal to a preset threshold T conf , it is considered that the current physical position of the special-shaped workpiece has a defect, and the defect detection result is directly outputted, and the defect recognition is completed;

[0023] If all the preliminary detection results of the special-shaped workpiece images do not detect defects, or the confidence of the position box of the defect category in the preliminary detection result is lower than the preset threshold T conf , it is considered that the current physical position of the special-shaped workpiece does not have a defect, and the step S5 is continued to be executed.

[0024] Accordingly, in the rough defect detection stage, the strategy of "detecting high-confidence defects in any image to directly determine" is adopted, so that in the detection scene with high safety level, the macro defect missed detection is prevented to the greatest extent, so as to ensure the production safety and quality stability.

[0025] At the same time, through the rapid screening in step S4, the special-shaped workpiece with obvious defects is directly rejected, and only the workpiece image suspected of having no defect or having a low-confidence defect is transmitted to the fine defect detection model, so that the calling frequency of high-computing segmentation detection is significantly reduced, and the overall detection efficiency and computing resource consumption are effectively balanced on the premise of ensuring the detection accuracy.

[0026] Further, the fine defect detection adopts a defect image segmentation model based on a sliding window, which specifically includes the following steps when detecting fine defects in any preprocessed special-shaped workpiece image:

[0027] Any preprocessed special-shaped workpiece image is cut by sliding according to a preset window size, to obtain a plurality of image blocks;

[0028] All the image blocks are sequentially inputted into the defect image segmentation model for processing, and the defect categories and confidence distributions of all the pixels in each image block are respectively outputted;

[0029] mapping the defect category and confidence distribution corresponding to the pixels in all image blocks to the image coordinate system of the current special-shaped workpiece image, and performing non-maximum suppression on the pixel confidence of the overlapping area to obtain a defect probability map of the current special-shaped workpiece image;

[0030] When the defect probability maps of all special-shaped workpiece images are obtained, the following judgment is performed:

[0031] If there is a pixel with a pixel value greater than or equal to a preset segmentation threshold T seg , and the connected area formed by the adjacent pixels reaches an area threshold or meets a shape feature condition, it is determined that there is a fine defect at the position;

[0032] Otherwise, it is considered that the current position of the current special-shaped workpiece does not have a defect, and the final defect detection result is output.

[0033] Accordingly, the present application introduces a defect image segmentation model based on a sliding window in the fine defect detection stage, and divides the entire special-shaped workpiece image into several smaller image blocks for analysis one by one, thereby significantly improving the resolution of micro cracks, fine scratches and other detail areas, and avoiding the loss of detail features due to the large scale of the entire image.

[0034] At the same time, by fusing the pixel confidence of the overlapping area of the sliding window and combining the connected area threshold and the shape feature condition for judgment, background noise and isolated false defects can be effectively suppressed, and the false detection rate can be reduced.

[0035] A defect detection device for large special-shaped metal workpieces, comprising a multi-position shooting image acquisition unit, an image preprocessing unit, a coarse defect detection model, a defect threshold judgment unit of a special-shaped workpiece and a fine defect detection model;

[0036] The multi-position shooting image acquisition unit is used to acquire multi-position shooting images of the special-shaped workpiece passing through the semi-closed light field assembly;

[0037] The multi-position shooting image includes a positive angle high exposure image, a positive angle low exposure image, a negative angle high exposure image and a negative angle low exposure image.

[0038] The image preprocessing unit is used to preprocess the multi-position shooting images of the special-shaped workpiece passing through the semi-closed light field assembly to obtain a plurality of preprocessed special-shaped workpiece images.

[0039] The coarse defect detection model is used to perform coarse defect detection on the plurality of preprocessed special-shaped workpiece images to obtain a plurality of preliminary detection results.

[0040] The defect threshold judging unit of the special-shaped workpiece is used for judging whether all the preliminary detection results are defect-free: if not, it is considered that the current position of the special-shaped workpiece has defects, and the defect recognition is completed; if yes, the fine defect detection model is called.

[0041] The fine defect detection model is used for fine defect detection on a plurality of preprocessed special-shaped workpiece images to obtain a final detection result.

[0042] A defect detection system for a stamping production line of automobile special-shaped workpieces comprises a conveying belt, a stamping die and a semi-enclosed light field assembly arranged above the conveying belt in sequence along a conveying direction, and a large special-shaped metal workpiece defect detection device as described above in communication connection with the semi-enclosed light field assembly.

[0043] The stamping die is used for stamping a metal plate into a special-shaped workpiece for automobiles, and the special-shaped workpiece is conveyed to a detection position of the semi-enclosed light field assembly by the conveying belt.

[0044] The semi-enclosed light field assembly is a light field cavity space, the top and the surrounding wall of the light field cavity space are closed, and the bottom is open and faces the conveying belt; wherein the inner surface of the light field cavity space is coated with a high diffuse reflection coating to form a uniform diffuse reflection light field without a specific incident angle.

[0045] The light field cavity is internally provided with a uniform illumination device and a plurality of high-speed shooting cameras, the uniform illumination device is used for providing illumination and making the light be reflected by the high diffuse reflection coating to form a uniformly illuminated light field.

[0046] The high-speed shooting cameras comprise a positive angle high exposure camera, a positive angle low exposure camera, a negative angle high exposure camera and a negative angle low exposure camera.

[0047] In order to better understand and implement, the present application is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 It is a structural schematic view of the defect detection system for automobile special-shaped workpieces.

[0049] Figure 2 It is a simple structural schematic view of the large special-shaped metal workpiece defect detection device.

[0050] Figure 3 It is a simple flow schematic view of the large special-shaped metal workpiece defect detection method. DETAILED DESCRIPTION

[0051] In order to solve the problems of low detail capturing ability of strong light reflection surface, weak defect detection performance and high missing detection rate in the high safety detection of large special-shaped metal workpieces in the prior art, the present application obtains multi-position shooting images of the special-shaped workpieces passing through the semi-closed light field assembly, pre-processes the images, inputs a plurality of pre-processed special-shaped workpiece images into a rough defect detection model for defect detection to obtain a plurality of preliminary detection results, then judges whether all the preliminary detection results are defect-free, if not, it is considered that the detection position of the current special-shaped workpiece has defects, and the defect detection is completed, if yes, the plurality of pre-processed special-shaped workpiece images are further input into a fine defect detection model for defect detection to obtain the final detection result, and the defect detection is completed.

[0052] Accordingly, the present application can realize high-quality imaging on the surface of large special-shaped workpieces with different curvatures and different inclination angles, effectively suppress the interference of strong light reflection on image details, and through two-stage defect detection, macro defects (rough defect detection) and fine defects (fine defect detection) are processed respectively, which can maximize the detection accuracy while minimizing the missing detection rate, thereby significantly improving the quality control level and safety of the automobile stamping production line, and ensuring stable and reliable defect recognition performance even under high-speed production conditions.

[0053] Based on the above design, the present application proposes a defect detection method for large special-shaped metal workpieces, and based on the method, a defect detection device for large special-shaped metal workpieces is proposed.

[0054] Please refer to Figure 1 , Figure 1 It is a structural schematic view of a defect detection system for automobile special-shaped workpieces.

[0055] A defect detection system for a stamping production line of automobile special-shaped workpieces, comprising a conveying belt, a stamping die arranged in sequence along the conveying direction, and a semi-closed light field assembly arranged above the conveying belt, and a defect detection device for large special-shaped metal workpieces in communication connection with the semi-closed light field assembly.

[0056] The conveying belt is used to convey the metal sheet along the conveying direction.

[0057] The stamping die is used to stamp the metal sheet to form a special-shaped workpiece for automobile, and the special-shaped workpiece is conveyed to the detection position of the semi-closed light field assembly by the conveying belt.

[0058] The semi-closed light field assembly is a light field cavity space, the top and the surrounding wall of the light field cavity space are closed, the bottom is open and faces the conveying belt, at the same time, the inner surface of the light field cavity space is coated with a high diffuse reflection coating to form a uniform diffuse reflection light field without specific incident angle.

[0059] Further, the entrance and exit of the light field cavity are provided with light shielding structures to weaken external light interference.

[0060] The inside of the light field cavity is provided with a uniform illumination device and a plurality of high-speed shooting cameras, the uniform illumination device is used to provide illumination and make the light reflected by the high-diffuse reflection coating to form a uniformly illuminated light field.

[0061] The high-speed shooting cameras are set to four by default, which are a positive angle high exposure camera, a positive angle low exposure camera, a negative angle high exposure camera and a negative angle low exposure camera.

[0062] The positive angle high exposure camera means that the inclination angle of the camera optical axis relative to the normal of the special-shaped workpiece is positive, and the default interval is [15°, 30°], which is used to shoot the special-shaped workpiece by using high exposure parameters, and transmit the shooting image to the defect detection device of the large special-shaped metal workpiece through communication.

[0063] The positive angle low exposure camera means that the inclination angle of the camera optical axis relative to the normal of the special-shaped workpiece is positive, and the default interval is [15°, 30°], which is used to shoot the special-shaped workpiece by using low exposure parameters, and transmit the shooting image to the defect detection device of the large special-shaped metal workpiece through communication.

[0064] The negative angle high exposure camera means that the inclination angle of the camera optical axis relative to the normal of the special-shaped workpiece is negative, and the default interval is [-15°, -30°], which is used to shoot the special-shaped workpiece by using high exposure parameters, and transmit the shooting image to the defect detection device of the large special-shaped metal workpiece through communication.

[0065] The negative angle low exposure camera means that the inclination angle of the camera optical axis relative to the normal of the special-shaped workpiece is negative, and the default interval is [-15°, -30°], which is used to shoot the special-shaped workpiece by using low exposure parameters, and transmit the shooting image to the defect detection device of the large special-shaped metal workpiece through communication.

[0066] Further, the semi-closed light field assembly further includes a position sensor and / or an encoder, the output of which is hardware trigger synchronization controlled by an I / O controller or a programmable logic controller (PLC) and the camera and the illumination device, so that the four shooting images of the same physical position are synchronously collected.

[0067] Therefore, compared with defect detection directly on the production line, the light field cavity space formed by the semi-closed light field assembly effectively eliminates the interference of specular reflection caused by a specific incident angle, and significantly reduces the influence of strong reflection on the capture of image details. At the same time, the semi-closed light field assembly cooperates with the uniform illumination device, so that the surfaces of the special-shaped workpieces at different positions and with different curvatures obtain consistent illumination conditions in the imaging process, reducing the local overexposure or underexposure phenomenon caused by the difference in surface inclination angle, thereby improving the stability of the model when extracting features from the captured images.

[0068] Please refer to Figure 2 and Figure 3 , Figure 2 for the simple structure diagram of the large special-shaped metal workpiece defect detection device described in the present application, Figure 3 for the simple flow diagram of the large special-shaped metal workpiece defect detection method described in the present application.

[0069] The large special-shaped metal workpiece defect detection device comprises a multi-position shooting image acquisition unit 1, an image preprocessing unit 2, a rough defect detection model 3, a special-shaped workpiece defect threshold judgment unit 4 and a fine defect detection model 5.

[0070] The multi-position shooting image acquisition unit 1 is used to perform step S1: acquiring multi-position shooting images of the special-shaped workpiece passing through the semi-closed light field assembly.

[0071] Among them, the multi-position shooting image includes a positive angle high exposure image, a positive angle low exposure image, a negative angle high exposure image and a negative angle low exposure image.

[0072] The present application acquires images of the special-shaped workpiece at the same physical position by adopting two different observation angles and two different exposure parameters. The different observation angles are mainly used for defects with directional reflection characteristics (such as scratches and indentation), because such defects may reflect strongly and have obvious outlines at a certain angle, but may not be easily detected at another angle. In addition, the different exposure parameters mainly capture the detail information of the dark area of the special-shaped workpiece surface by high exposure, and retain the texture details of the highlight area (such as local metal reflection points) by low exposure.

[0073] Finally, through the images shot at different observation angles and different exposure parameters, the present application effectively makes up for the information missing problem caused by the single camera shooting of the special-shaped workpiece, and can ensure that potential defect features can be captured under different surface reflection conditions, thereby reducing the risk of missed detection caused by factors such as illumination, angle and reflection.

[0074] The image preprocessing unit 2 is configured to perform step S2: preprocessing the multi-station shooting images of the special-shaped workpiece passing through the semi-closed light field assembly to obtain a plurality of preprocessed special-shaped workpiece images.

[0075] Specifically, the preprocessing includes geometric correction, image registration and size transformation. The geometric correction is used to correct the geometric distortion of all multi-station shooting images, thereby eliminating the radial distortion or tangential distortion caused by the lens and ensuring the size accuracy of the defect position in the subsequent detection. The geometric distortion correction can be realized according to the calibration data of the camera, for example, the camera intrinsic parameters and distortion coefficients are calculated by using the images collected by the calibration board, and the original image is corrected by pixel remapping, and the camera intrinsic parameters and distortion coefficients are different according to different devices and scenes, which are not limited in the present application.

[0076] The image registration is used to align the space position of all images after geometric correction, so that the shooting images of different stations are mapped to the same reference plane coordinate system, and the workpiece regions at the same physical position are one-to-one corresponding in multiple images, thereby improving the effectiveness of multi-angle and multi-exposure information utilization.

[0077] The size transformation is used to adjust the scale of all images after image registration to make them have a unified input size, thereby obtaining a plurality of preprocessed special-shaped workpiece images.

[0078] The size transformation can be any form such as scaling, cropping, padding, etc., and the specific method can be set according to the input specification, computing resources or detection accuracy requirements during model training, which is not limited in the present application.

[0079] Accordingly, through the above preprocessing, the geometric differences caused by the shooting position deviation or the transmission posture change of the special-shaped workpiece can be effectively eliminated, and the spatial reference and size specification of the images are unified, thereby improving the performance of the subsequent defect detection model in terms of detection accuracy and stability. In addition, the specific implementation method of the preprocessing can be adjusted according to the actual production environment and device conditions, which is not limited in the present application.

[0080] The rough defect detection model 3 is configured to perform step S3: rough defect detection on a plurality of preprocessed special-shaped workpiece images to obtain a plurality of preliminary detection results.

[0081] Specifically, the rough defect detection model 3 adopts a defect detection model based on the YOLO (You Only Look Once) series, which includes a backbone network, a neck network and a detection head.

[0082] The backbone network is configured to perform multi-scale feature extraction on any preprocessed special-shaped workpiece image and output a plurality of feature maps of different scales.

[0083] The backbone network can be any deep convolutional neural network structure such as CSPDarknet, ResNet or MobileNet, so as to balance the feature expression ability of the feature map and the inference speed, and the present application does not make specific limitation thereto.

[0084] The neck network is configured to perform feature fusion on feature maps of different scales and output a plurality of fused feature maps.

[0085] The neck network can adopt a feature pyramid (Feature Pyramid Network, FPN) or a path aggregation network (Path Aggregation Network, PANet), and the present application does not make specific limitation thereto.

[0086] The detection head is configured to classify and regress the fused feature maps of different scales, output defect categories, position box coordinates and confidence, and form a preliminary detection result of the current special-shaped workpiece image.

[0087] The detection head adopts an Anchor-based structure or an Anchor-free structure of the YOLO series, includes parallel classification layers and regression layers, the classification layers are configured to determine defect categories, and the regression layers are configured to predict center coordinates, width, height and other parameters of a position box of a defect, and the present application does not make specific limitation to the specific structure of the detection head.

[0088] Accordingly, each preprocessed special-shaped workpiece image is independently detected to obtain a preliminary detection result corresponding to each machine position, so that each preliminary detection result is subjected to secondary determination, so as to determine whether to call a subtle defect detection model, thereby reducing unnecessary computing overhead.

[0089] The defect threshold judgment unit 4 of the special-shaped workpiece is configured to perform step S4: judging whether all preliminary detection results are defect-free: if not, it is considered that the current position of the special-shaped workpiece has defects, the defect recognition is completed, and manual intervention is notified or the current special-shaped workpiece is transferred to a defect classification area; if yes, the subtle defect detection model 5 is called.

[0090] Specifically, if there is a position box of a defect category in the preliminary detection result of any special-shaped workpiece image, and the confidence of the position box is greater than or equal to a preset threshold T conf , it is considered that the current physical position of the special-shaped workpiece has defects, and the defect detection result is directly outputted, and the defect recognition is completed.

[0091] If all the preliminary detection results of the special-shaped workpiece images do not detect defects, or the confidence of the position box of the defect category of the preliminary detection result is lower than the preset threshold T conf , it is considered that the current physical position of the special-shaped workpiece does not exist defects, and the fine defect detection model 5 is called for re-inspection.

[0092] Wherein, the preset threshold T conf may be preset according to the safety level requirement of the actual application scene, and the present application defaults to 0.85, and the preset threshold is not specifically limited.

[0093] Accordingly, when defects are detected in the preliminary detection results of the special-shaped workpiece images taken at any one position and the confidence condition is met, it can be directly determined that defects exist at this position, thereby preventing missed detection to the greatest extent and significantly improving the quality control and safety guarantee level of the stamping production process.

[0094] The fine defect detection model 5 is used to perform step S5: fine defect detection on a plurality of preprocessed special-shaped workpiece images to obtain a final detection result.

[0095] Specifically, the fine defect detection model is a defect image segmentation model based on a sliding window, and the defect image segmentation model defaults to YOLOv8-seg, that is, the pixel-level segmentation of defects is realized by combining the detection head with the mask.

[0096] The defect image segmentation model based on the sliding window, when performing fine defect detection on any one of the preprocessed special-shaped workpiece images, specifically includes performing the following steps:

[0097] First, any one of the preprocessed special-shaped workpiece images is cut by sliding according to a preset window size to obtain a plurality of image blocks;

[0098] Wherein, the preset window size defaults to 512x512 pixels, and the sliding step defaults to 1 / 4 to 1 / 2 of the preset window size, which can be adjusted according to actual conditions, and the present application does not specifically limit the selection of the preset window size and the sliding step.

[0099] Next, all the image blocks are sequentially input into the defect image segmentation model for processing, and the defect category and confidence distribution of all pixels in each image block are output respectively.

[0100] Wherein, the defect image model includes a backbone network, a neck network and a segmentation detection head; the backbone network and the neck network can be the same as the defect detection model of the YOLO series used for rough defect detection as described above by default, and are used for multi-scale feature extraction and fusion, which will not be described in detail herein.

[0101] The segmentation probe head is used for pixel-level classification on the multi-scale fusion feature map output by the neck network to obtain a defect category and a confidence distribution corresponding to all pixels in the image block.

[0102] Then, the defect category and the confidence distribution corresponding to the pixels in all image blocks are mapped to the image coordinate system of the current irregular workpiece image, and the confidence of the pixels in the overlapping area is suppressed by non-maximum suppression to obtain a defect probability map of the current irregular workpiece image.

[0103] The defect probability map is used to represent the defect category and the confidence value corresponding to each pixel of the current irregular workpiece image.

[0104] Finally, when the defect probability maps of all irregular workpiece images are obtained, the following judgment is performed:

[0105] If there is a pixel with a pixel value greater than or equal to a preset segmentation threshold T seg , and the connected area formed by adjacent pixels reaches an area threshold or meets a shape feature condition, it is determined that there is a subtle defect at the position.

[0106] Otherwise, it is considered that the current position of the current irregular workpiece does not have a defect, and the final defect detection result is output.

[0107] The preset segmentation threshold T seg is 0.8 by default, and can be adjusted according to different safety levels, which is not limited in the present application; the area threshold represents the lower limit value of the number of pixels of the connected area with a pixel confidence greater than or equal to the preset segmentation threshold T seg in the defect probability map, which is used to filter false detection areas caused by noise or texture details, and the default area threshold is 50 pixels, which can be adjusted according to different detection resolutions and workpiece surface complexities, and the present application does not limit the specific area threshold.

[0108] The shape feature condition represents that the geometric characteristics of the connected area with a pixel confidence greater than or equal to the preset segmentation threshold T seg in the defect probability map meet specific morphological rules, such as aspect ratio, edge curvature, roundness or convexity, which are mainly used to filter meaningless areas caused by random reflections, burrs or scratches, and the present application does not limit the specific shape feature condition, and users can define the rules according to actual production line defect types.

[0109] Accordingly, by performing pixel-level segmentation detection in a local high-resolution window and mapping and fusing the results to the entire image, defects with small size, irregular shape or low contrast can be effectively identified, the probability of missed detection is significantly reduced, and the detection accuracy and safety of the stamping production line for large irregular metal workpieces are improved.

[0110] Further, the final defect detection results are archived for subsequent quality tracing and statistical analysis.

[0111] Specifically, each final defect detection result is stored in one-to-one correspondence with the physical position, production time and production batch information of the corresponding profiled workpiece to form a queryable defect record database.

[0112] The physical position can be an absolute coordinate position or a relative position identifier on the stamping production line, and the production batch information can be automatically obtained from a production management system or manually input by an operator, and the information collection method is not specifically limited by the application.

[0113] In addition, according to different profiled workpiece types or material types, corresponding classification archives can be constructed to realize category-based retrieval, statistical defect distribution and trend analysis, and the data storage method can be a local database, a cloud storage service or an embedded storage module of a production management system, and the application does not specifically limit this.

[0114] Accordingly, by associating the final defect detection results with the production and position information of the profiled workpiece, full-process quality tracing can be realized, which facilitates quick positioning of the causes and links of defects in subsequent quality management, customer feedback processing or process improvement, thereby further improving production quality and management efficiency.

[0115] Compared with the prior art, the application sets a semi-enclosed light field assembly on the stamping production line and coats the inner surface with a high diffuse reflection coating, so that the incident light is uniformly reflected multiple times in the cavity, thereby forming a diffuse reflection light field without a specific incident angle, effectively eliminating the strong reflection phenomenon of large profiled metal workpiece surfaces caused by curvature changes and inclination angle differences; combined with multi-position and different exposure parameter shooting modes, the surface detail capture capability is further improved, providing a high-quality image basis for subsequent defect identification.

[0116] Secondly, the application adopts a rough defect detection model to preliminarily detect the multi-position images, and introduces a strict conditional logic: when there is a defect frame with a confidence higher than a preset threshold in any one of the detection results, it is directly determined that there is a defect and the result is output; only when all the detection results are lower than the threshold, the fine defect detection stage is entered, realizing the quick elimination of macro defects and minimizing the missed detection probability through the high-confidence triggering mechanism.

[0117] Finally, in the fine defect detection stage, the application adopts a defect image segmentation model based on a sliding window (default YOLOv8-seg) to perform pixel-level segmentation on the image, maps the detection results of each window back to the original image, fuses to generate a defect probability map, and then determines through a segmentation threshold and a connected domain area or shape feature condition, so that low-contrast defects such as superficial cracks and micro scratches can be effectively identified, the detection capability under background noise interference is improved, and thus fine defects on the complex surface structure of the special-shaped workpiece can also be effectively identified.

[0118] Based on the same inventive concept, the application also provides an electronic device, which can be a server, a desktop computing device or a mobile computing device (for example, a laptop computer, a handheld computing device, a tablet computer, a netbook, etc.) and the like terminal device. The device includes one or more processors and a memory, wherein the processor is used to execute a program to realize the defect detection method of the large special-shaped metal workpiece; the memory is used to store the computer program executable by the processor.

[0119] Based on the same inventive concept, the application also provides a computer readable storage medium, corresponding to the above-mentioned embodiments of the defect detection method of the large special-shaped metal workpiece, the computer readable storage medium has a computer program stored thereon, and the program is executed by the processor to realize the steps of the defect detection method of the large special-shaped metal workpiece described in any of the above embodiments.

[0120] The application can adopt the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. The computer usable storage medium includes permanent and non-permanent, removable and non-removable media, and can be realized by any method or technology. Information storage. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0121] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and the present application also intends to include these modifications and improvements.

Claims

1. A defect detection method for large special-shaped metal workpieces, characterized in that: The following steps are involved: S1: Acquire multi-camera images of a special-shaped workpiece passing through a semi-enclosed light field component; The multi-camera shooting images include positive angle high exposure images, positive angle low exposure images, negative angle high exposure images and negative angle low exposure images; S2: Preprocessing the multi-camera images of the special-shaped workpiece passing through the semi-enclosed light field component to obtain a number of preprocessed special-shaped workpiece images; S3: Perform rough defect detection on several pre-processed images of special-shaped workpieces to obtain several preliminary detection results; S4: Determine whether all preliminary inspection results show that there is no defect: if not, it is considered that there is a defect at the current position of the special-shaped workpiece and the defect identification is completed; if so, execute step S5; S5: Perform subtle defect detection on several pre-processed images of special-shaped workpieces to obtain final detection results.

2. The defect detection method for large special-shaped metal workpieces according to claim 1, characterized in that: The coarse defect detection is performed by a coarse defect detection model, which includes a backbone network, a neck network and a detection head; The backbone network is used to perform multi-scale feature extraction on any pre-processed irregular workpiece image and output feature maps of several different scales; The neck network is used to perform feature fusion on several feature maps of different scales and output several fused feature maps; The detection head is used to classify and regress fused feature maps of different scales, output defect categories, location frame coordinates and confidence levels, and form preliminary detection results for the current irregular workpiece image; The determination in step S4 further includes the following sub-steps: If there is a position frame of the defect category in the preliminary detection result of any special-shaped workpiece image, and the confidence of the position frame is greater than or equal to a preset threshold T conf , it is considered that there is a defect in the current physical position of the special-shaped workpiece, and the defect detection result is directly output to complete the defect identification; If no defects are detected in the preliminary inspection results of all special-shaped workpiece images, or the confidence of the position box of the defect category in the preliminary inspection results is lower than the preset threshold T conf , it is considered that there is no defect in the current physical position of the special-shaped workpiece, and step S5 is continued.

3. The defect detection method for large special-shaped metal workpieces according to claim 2, characterized in that: The subtle defect detection adopts a defect image segmentation model based on a sliding window. When performing subtle defect detection on any pre-processed irregular workpiece image, it specifically includes executing the following steps: Sliding crop any pre-processed irregular workpiece image according to the preset window size to obtain several image blocks; All image blocks are sequentially input into the defect image segmentation model for processing, and the defect category and confidence distribution of all pixels in each image block are output respectively; The defect categories and confidence distributions corresponding to the pixels in all image blocks are mapped to the image coordinate system of the current irregular workpiece image, and the pixel confidences in the overlapping areas are non-maximum suppressed to obtain the defect probability map of the current irregular workpiece image. After obtaining the defect probability maps of all special-shaped workpiece images, the following judgments are made: If there is a pixel value greater than or equal to a preset segmentation threshold T seg If the pixel is , and the area of ​​the connected region formed by the adjacent pixels reaches an area threshold or meets the shape feature condition, it is determined that there is a subtle defect at that location; Otherwise, it is considered that there is no defect at the current position of the current special-shaped workpiece, and the final defect detection result is output.

4. A defect detection device for large special-shaped metal workpieces, characterized in that: It includes a multi-camera image acquisition unit, an image preprocessing unit, a rough defect detection model, a defect threshold judgment unit for special-shaped workpieces, and a subtle defect detection model; The multi-camera image acquisition unit is used to acquire multi-camera images of the special-shaped workpiece passing through the semi-enclosed light field component; The multi-camera shooting images include positive angle high exposure images, positive angle low exposure images, negative angle high exposure images and negative angle low exposure images; The image preprocessing unit is used to preprocess the multi-camera images of the special-shaped workpiece passing through the semi-enclosed light field component to obtain a plurality of preprocessed special-shaped workpiece images; The rough defect detection model is used to perform rough defect detection on a number of pre-processed images of special-shaped workpieces to obtain a number of preliminary detection results; The defect threshold judgment unit of the special-shaped workpiece is used to judge whether all preliminary detection results show that there is no defect: if not, it is considered that there is a defect at the current position of the special-shaped workpiece and the defect identification is completed; if so, the subtle defect detection model is called; The subtle defect detection model is used to perform subtle defect detection on a number of pre-processed images of special-shaped workpieces to obtain final detection results.

5. The defect detection device for large special-shaped metal workpieces according to claim 4, characterized in that: The rough defect detection model includes a backbone network, a neck network and a detection head; The backbone network is used to perform multi-scale feature extraction on any pre-processed irregular workpiece image and output feature maps of several different scales; The neck network is used to perform feature fusion on several feature maps of different scales and output several fused feature maps; The detection head is used to classify and regress fused feature maps of different scales, output defect categories, location frame coordinates and confidence levels, and form preliminary detection results for the current irregular workpiece image; The defect threshold determination unit for the special-shaped workpiece further includes executing the following sub-steps: If there is a position frame of the defect category in the preliminary detection result of any special-shaped workpiece image, and the confidence of the position frame is greater than or equal to a preset threshold T conf , it is considered that there is a defect in the current physical position of the special-shaped workpiece, and the defect detection result is directly output to complete the defect identification; If no defects are detected in the preliminary inspection results of all special-shaped workpiece images, or the confidence of the position box of the defect category in the preliminary inspection results is lower than the preset threshold T conf , it is considered that there is no defect at the current physical position of the special-shaped workpiece, and the subtle defect detection model is called.

6. The defect detection device for large special-shaped metal workpieces according to claim 5, characterized in that: When the subtle defect detection model performs subtle defect detection on any pre-processed irregular-shaped workpiece image, it specifically includes executing the following steps: Sliding crop any pre-processed irregular workpiece image according to the preset window size to obtain several image blocks; All image blocks are sequentially input into the defect image segmentation model for processing, and the defect category and confidence distribution of all pixels in each image block are output respectively; The defect categories and confidence distributions corresponding to the pixels in all image blocks are mapped to the image coordinate system of the current irregular workpiece image, and the pixel confidences in the overlapping areas are non-maximum suppressed to obtain the defect probability map of the current irregular workpiece image. After obtaining the defect probability maps of all special-shaped workpiece images, the following judgments are made: If there is a pixel value greater than or equal to a preset segmentation threshold T seg If the pixel is , and the area of ​​the connected region formed by the adjacent pixels reaches an area threshold or meets the shape feature condition, it is determined that there is a subtle defect at that location; Otherwise, it is considered that there is no defect at the current position of the current special-shaped workpiece, and the final defect detection result is output.

7. A defect detection system for a stamping production line of automobile special-shaped workpieces, characterized in that: A device for detecting defects in large-scale special-shaped metal workpieces according to any one of claims 4 to 6, comprising a conveyor belt, a stamping die, and a semi-enclosed light field component arranged in sequence along a conveying direction and disposed above the conveyor belt, and the semi-enclosed light field component being communicatively connected thereto; The stamping die is used to stamp metal sheets into special-shaped workpieces for automobiles, and the conveyor belt conveys the special-shaped workpieces to the detection position of the semi-enclosed light field assembly; The semi-enclosed light field component is a light field cavity space, the top and surrounding walls of the light field cavity space are closed, and the bottom is open towards the conveyor belt; wherein the inner surface of the light field cavity space is coated with a high diffuse reflection coating to form a uniform diffuse reflection light field without a specific incident angle; The interior of the light field cavity is provided with a uniform lighting device and a plurality of high-speed shooting cameras, wherein the uniform lighting device is used to provide illumination and reflect the light through the high diffuse reflective coating to form a uniformly illuminated light field; The high-speed shooting camera includes a positive angle high exposure camera, a positive angle low exposure camera, a negative angle high exposure camera and a negative angle low exposure camera.

8. The defect detection system for a stamping production line of automobile special-shaped workpieces according to claim 7, characterized in that: The positive-angle high-exposure camera indicates that the inclination angle of the camera optical axis relative to the normal of the special-shaped workpiece is positive, and is used to photograph the special-shaped workpiece using high exposure parameters and transmit the photographed image to the defect detection device of the large special-shaped metal workpiece via communication; The positive-angle low-exposure camera indicates that the inclination angle of the camera optical axis relative to the normal of the special-shaped workpiece is positive, and is used to photograph the special-shaped workpiece using low exposure parameters and transmit the photographed image to the defect detection device of the large special-shaped metal workpiece via communication; The negative-angle high-exposure camera indicates that the inclination angle of the camera optical axis relative to the normal of the special-shaped workpiece is negative, and is used to photograph the special-shaped workpiece using high exposure parameters and transmit the photographed image to the defect detection device of the large special-shaped metal workpiece via communication; The negative-angle low-exposure camera indicates that the inclination angle of the camera optical axis relative to the normal of the special-shaped workpiece is negative, and is used to photograph the special-shaped workpiece using low exposure parameters, and transmit the photographed image to the defect detection device for the large special-shaped metal workpiece through communication.

9. An electronic device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for detecting defects in a large, special-shaped metal workpiece as described in any one of claims 1 to 3 is implemented.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by the processor, a defect detection method for a large special-shaped metal workpiece as described in any one of claims 1 to 3 is implemented.

Images