A method, device and system for quality inspection of stamping parts on a production line

Abstract

The present invention provides a method and device for quality inspection of stamping parts on a production line, including the following steps: constructing an object detection model based on a neural network through stamping part defect images; transmitting the stamping parts to be detected to the detection position through the production line transmission equipment; collecting image data of the high-risk points of defects of the stamping parts to be detected; and performing defect detection on the image data of the high-risk points of defects of the stamping parts to be detected according to the object detection model. The present invention also provides a quality inspection system for stamping parts on a production line.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and more particularly to the field of automated stamping part defect detection. Background Art

[0002] At present, the on-line quality inspection of stamping parts on the production line mainly relies on manual inspection. Due to the production rhythm limitation during stamping part production, 100% inspection cannot be achieved. The defects of stamping parts mainly include cracking, thinning, wrinkling, etc. Manual inspection has low efficiency and is prone to missed inspection, resulting in the risk of defective products flowing out. Therefore, this problem needs to be solved. Summary of the Invention

[0003] In order to solve the technical problems of low inspection efficiency and easy missed inspection in the inspection of stamping part production, the present invention proposes a method for quality inspection of stamping parts on a production line, including the following steps:

[0004] Construct a target detection model based on a neural network through stamping part defect images;

[0005] Transmit the stamping part to be detected to the detection position through the production line transmission equipment;

[0006] Collect image data of high-risk points of defects of the stamping part to be detected;

[0007] Perform defect detection on the image data of high-risk points of defects of the stamping part to be detected according to the target detection model.

[0008] In the above method for quality inspection of stamping parts on a production line, the step of constructing a target detection model based on a neural network through stamping part defect images further includes:

[0009] Construct a backbone network layer, a neck layer, and a head layer;

[0010] Replace the feature processing module at the last stage of the backbone network layer with a serpentine convolution module;

[0011] Replace all feature splicing modules of the neck layer with a weighted bidirectional feature pyramid network module.

[0012] In the above technical solution, serpentine convolution can better capture the boundaries of non-rigid targets, the weighted bidirectional feature pyramid network can further effectively fuse multi-scale features, and by introducing bidirectional connections, the ability of feature fusion is enhanced, improving the detection effect of stamping part defects.

[0013] In the above method for quality inspection of stamping parts on a production line, the step of constructing a target detection model based on a neural network through stamping part defect images further includes:

[0014] Perform annotations on the stamping part defect images including crack types and crack mouth types;

[0015] Data augmentation is performed on the labeled image to construct a training dataset.

[0016] In the above technical solution, by labeling the crack type and crack opening type of the defect image, the cracking of the stamping part can be accurately identified.

[0017] In the above method for detecting the quality of stamping parts on the production line, the step of performing data augmentation on the labeled image further includes:

[0018] Performing image augmentation on the labeled image includes mixing, mosaic, and salt-and-pepper noise.

[0019] In the above method for detecting the quality of stamping parts on the production line, the step of transferring the stamping part to be detected to the detection position through the production line transfer device further includes:

[0020] The programmable logic controller of the production line transfer device generates a signal indicating arrival.

[0021] In the above method for detecting the quality of stamping parts on the production line, the step of collecting the image data of the high-risk defect points of the stamping part to be detected further includes:

[0022] After detecting the signal indicating arrival, the image acquisition device collects the image data of multiple high-risk defect points of the stamping part to be detected.

[0023] In the above method for detecting the quality of stamping parts on the production line, the step of performing defect detection on the image data of the high-risk defect points of the stamping part to be detected according to the target detection model further includes:

[0024] Using the non-maximum suppression method to delete the detection frames with large overlap degrees and output an image with detection frames and defect category information.

[0025] In the above method for detecting the quality of stamping parts on the production line, the method further includes:

[0026] Saving the result of the defect detection;

[0027] Performing real-time visual display on the result of the defect detection;

[0028] Issuing an alarm when the result of the defect detection is abnormal.

[0029] In the above technical solution, the stamping part defects are displayed in a visual manner, the stamping part defects are alarmed in real time, and the stamping part defects can be reviewed in a timely manner.

[0030] To better achieve the object of the present invention, the present invention further provides a device for detecting the quality of stamping parts on a production line, which is used to implement any one of the above methods, and includes:

[0031] A target detection model construction module, configured to construct a neural network-based target detection model through stamping part defect images;

[0032] A transfer module, configured to transfer the stamping part to be detected to the detection position through the production line transfer equipment;

[0033] An image data acquisition module, configured to acquire image data of high-risk points of defects of the stamping part to be detected;

[0034] A defect detection module, configured to perform defect detection on the image data of high-risk points of defects of the stamping part to be detected according to the target detection model.

[0035] To better achieve the object of the present invention, the present invention also provides a quality detection system for stamping parts on a production line, including:

[0036] A stamping device, configured to stamp production materials to generate stamping parts to be detected;

[0037] A transfer device, configured to transfer the stamping part to be detected to the detection area;

[0038] A production line transfer equipment, arranged in the detection area, configured to transfer the stamping part to be detected to the detection position;

[0039] An image acquisition device, configured to acquire image data of high-risk points of defects of the stamping part to be detected;

[0040] A quality detection unit, connected to the production line transfer equipment and the image acquisition device, and the quality detection unit further includes the quality detection device for stamping parts on a production line as described above.

[0041] To better achieve the object of the present invention, the present invention also provides a storage medium, configured to store a computer control program, and the computer control program is used to execute the steps of any one of the above methods.

[0042] Compared with the prior art, a quality detection method, device and system for stamping parts on a production line provided by the present invention construct a target recognition model based on a neural network to automatically identify defects of stamping parts instead of manual labor; and optimize the target recognition model to better capture the boundaries of non-rigid targets, further effectively fuse multi-scale features, enhance the ability of feature fusion, and improve the detection effect of stamping part defects; display stamping part defects in a visual manner, alarm in real time for stamping part defects and can promptly review stamping part defects.

[0043] For a further understanding of the features and technical content of the present invention, please refer to the following detailed description and diagrams of the present invention. However, the provided diagrams are only for reference and illustration, and are not used to limit the present invention. Description of the Drawings

[0044] Figure 1 Shows a flowchart of the method for detecting the quality of stamping parts on a production line in an embodiment of the present invention.

[0045] Figure 2 Shows an architecture diagram of an object detection model in the prior art.

[0046] Figure 3 Shows an architecture diagram of the optimized object detection model in an embodiment of the present invention.

[0047] Figure 4 Shows a schematic diagram of feature fusion of the weighted bidirectional feature pyramid network module in an embodiment of the present invention.

[0048] Figure 5 Shows a block diagram of the device for detecting the quality of stamping parts on a production line in an embodiment of the present invention.

[0049] Figure 6 Shows a block diagram of the system for detecting the quality of stamping parts on a production line in an embodiment of the present invention.

[0050] Figure 7 Shows a block diagram of the system for detecting the quality of stamping parts on a production line in another embodiment of the present invention.

[0051] Figure 8 Shows a top view of defect identification of the stamping parts to be detected on the production line transmission equipment in an embodiment of the present invention.

[0052] Figure 9 Shows a front view of defect identification of the stamping parts to be detected on the production line transmission equipment in an embodiment of the present invention.

[0053] Figure 10 Shows a schematic diagram of real-time visual display of the defect detection results of the stamping parts to be detected in an embodiment of the present invention.

[0054] Figure 11 Shows a schematic diagram of multiple image acquisition devices for image acquisition of different types of stamping parts in an embodiment of the present invention.

[0055] Among them, reference numerals:

[0056] S1-S4... steps

[0057] 1... backbone network layer

[0058] 2... neck layer

[0059] 3... head layer

[0060] 4... image data of high-risk points of defects of the stamping parts to be detected

[0061] 5… Serpentine Convolution Module

[0062] 6… Weighted Bidirectional Feature Pyramid Network Module

[0063] 10… Production Line Stamping Part Quality Detection Device

[0064] 11… Target Detection Model Construction Module

[0065] 12… Transfer Module

[0066] 13… Image Data Acquisition Module

[0067] 14… Defect Detection Module

[0068] 100… Production Line Stamping Part Quality Detection System

[0069] 110… Stamping Equipment

[0070] 120… Transfer Device

[0071] 130… Production Line Transmission Equipment

[0072] 131… Programmable Logic Controller

[0073] 140… Image Acquisition Equipment

[0074] 150… Quality Detection Unit

[0075] 151… Server

[0076] 160… Stamping Parts to be Detected

[0077] 170… Support Frame

[0078] 171… Upper Frame

[0079] 172… Lower Frame

[0080] 173… First Support Rod

[0081] 174… Second Support Rod

[0082] 175… Light Emitting Unit

[0083] 180… Visualization Unit Detailed Implementation Manner

[0084] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments, so as to further understand the purpose, solution and beneficial technical effects of the present invention. Obviously, the specific embodiments described herein are only a part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments disclosed in the present invention without creative efforts also belong to the scope disclosed by the technical solution of the present invention.

[0085] It should be noted that in this specification, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, article or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0086] In the specification and the appended claims, certain terms are used to refer to specific components or parts. Those of ordinary skill in the art should understand that different nouns or terms may be used by technical users or manufacturers to refer to the same component or part. The specification and the appended claims do not use the difference in names as a way to distinguish components or parts, but use the difference in the functions of components or parts as the criterion for distinction.

[0087] In the present invention, the orientation or positional relationship indicated by terms such as "upper", "lower", "left", "right", "front", "rear", "top", "bottom", "inner", "outer", "middle", "vertical", "horizontal", "lateral", "longitudinal", etc. is based on the orientation or positional relationship shown in the drawings. These terms are mainly used to better describe the present invention and its embodiments, and are not used to limit that the indicated device, element or component must have a specific orientation or be constructed and operated in a specific orientation.

[0088] In addition, the terms "installed", "set", "provided with", "connected", "connected to" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or there is internal communication between two devices, elements or components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0089] To better understand the technical solution of the present invention, reference can be made to Figure 1 In order to solve the technical problems of low inspection efficiency and easy omission of inspection in the production of stamping parts, the present invention proposes a method for quality inspection of stamping parts on a production line, including the following steps:

[0090] Step S1: Construct an object detection model based on a neural network through stamping part defect images;

[0091] Step S2: Transmit the stamping part to be detected to the detection position through the production line transmission equipment;

[0092] Step S3: Collect image data 4 of the high-risk points of defects of the stamping part to be detected;

[0093] Step S4: Perform defect detection on the image data 4 of the high-risk points of defects of the stamping part to be detected according to the object detection model.

[0094] In an implementation of the method for quality inspection of stamping parts on the production line of the present invention, the step of constructing an object detection model based on a neural network through stamping part defect images further includes:

[0095] Construct a backbone network layer (backbone) 1, a neck layer (neck) 2, and a head layer (head) 3;

[0096] Use a dilated separable convolution module (DSConv) 5 to replace the feature processing module at the last stage of the backbone network layer 1;

[0097] Use a weighted bidirectional feature pyramid network module (BiFPN) 6 to replace all the feature concatenation (Concat) modules of the neck layer 2;

[0098] The loss function used in the object detection model is divided into prediction box loss, object confidence loss, and defect category loss. Among them, the prediction box loss function is EIOU Loss, and the object confidence loss and defect category loss functions are binary cross-entropy functions with sigmoid, BCEWithLogitsLoss.

[0099] Specifically, to better understand the technical solution of the present invention, reference can be made to Figure 2 and Figure 3 , Figure 2 shows the architecture diagram of the object detection model in the prior art, Figure 3The architecture diagram of the optimized object detection model in an embodiment of the present invention is shown. When constructing an object detection model based on a neural network, any one of the yolov5 series of object detection models can be selected. The backbone network layer 1, neck layer 2, and head layer 3 of the yolov5 series of object detection models can perform object recognition on the input image data of defect high-risk points, and the component modules of each layer are all prior arts, which will not be elaborated herein.

[0100] To better detect the defects of stamping parts, the object detection model is optimized, and the serpentine convolution module 5 is used to replace the feature processing module (C3 module) at the last stage of the backbone network layer 1;

[0101] Regarding the principle of serpentine convolution:

[0102] Given a standard 2D convolution coordinate K with the center coordinate K i =(x i +y i ), a 3×3 convolution kernel K is expressed as:

[0103] K ={(x - 1,y - 1),(x - 1,y),…,(x + 1,y + 1)} (1)

[0104] To endow the convolution kernel with more flexibility so that it can focus on the complex geometric features of the target, the deformation offset Δ is introduced. The dynamic serpentine convolution linearizes the standard convolution kernel in both the x-axis and y-axis directions. Considering a convolution kernel of size 9, taking the x-axis direction as an example, the specific position of each grid in K is expressed as: K i±c =(x i±c +y i±c ), where c = 0,1,2,3,4 represents the horizontal distance from the central grid. The selection of each grid position K i±c in the convolution kernel K is an accumulative process. Starting from the central position K i , the position far from the central grid depends on the position of the previous grid: K i±1 relative to K i increases the offset Δ ={δ|δ∈[-1,1]}. Therefore, the offset needs to be accumulated Σ to ensure that the convolution kernel conforms to the linear morphological structure. The change in the x-axis direction is:

[0105]

[0106] The change in the y-axis direction is:

[0107]

[0108] Since the offset Δ is usually a decimal, and the coordinates are usually in integer form, bilinear interpolation is adopted, expressed as:

[0109] K = ∑ K′ B(K′, K)·K′ (4)

[0110] Wherein, K represents the decimal position of Equation (2) and Equation (3), K′ enumerates all integer spatial positions, B is a bilinear interpolation kernel, which can be decomposed into two one-dimensional kernels, that is:

[0111] B(K′, K) = b(K x , K′ x )·b(K y , K′ y ) (5)

[0112] Replace all feature concatenation (Concat) modules in Neck Layer 2 with a weighted bidirectional feature pyramid network module. BiFPN is an improved feature pyramid network (FPN) structure. Through bidirectional connections and adaptive feature adjustment, it improves the performance of object detection and semantic segmentation tasks, enabling the neural network to better understand and interpret multi-scale information, and improving the detection accuracy of small objects. As Figure 4 shown, introducing BiFPN in yolov5 can more effectively detect small target objects on stamping parts. Small targets usually occupy fewer pixels in the image, so they are more likely to be ignored or misjudged.

[0113] The introduction of BiFPN allows information to propagate bidirectionally between different resolution levels, thus better fusing multi-scale information. This helps the model to more comprehensively understand targets of different sizes, improving the detection performance for multi-scale objects. At the same time, it also improves the context understanding of objects, helping to reduce false positives or false negatives.

[0114] In the above technical solution, non-rigid target boundaries can be better captured through serpentine convolution. The weighted bidirectional feature pyramid network can further effectively fuse multi-scale features, and by introducing bidirectional connections, the ability of feature fusion is enhanced, improving the detection effect of stamping part defects.

[0115] In an implementation of the stamping part quality detection method of the present invention, the steps of constructing an object detection model based on a neural network through stamping part defect images further include:

[0116] Label the stamping part defect images including crack types and crack mouth types;

[0117] Perform data augmentation on the labeled images to construct a training dataset.

[0118] In the above technical solution, stamping part cracking can be accurately identified by labeling the defect images with crack types and crack mouth types.

[0119] In an implementation of the method for detecting the quality of stamping parts in the production line of the present invention, the step of data augmentation on the labeled image further includes:

[0120] Image augmentation on the labeled image includes mixing, mosaic, and salt-and-pepper noise.

[0121] Specifically, use labelimg to label the stamping part defect images. The labeled defect categories are at least divided into two categories: cracks and fissures. Then, use one or more of the methods of mixing (mixup), mosaic, and salt-and-pepper noise for data augmentation to produce a training data set. Mixing: Randomly select two images from each batch and mix them in a certain proportion to generate a new image. It should be noted that only the new mixed images are used for training throughout the entire training process, and the original images do not participate in the training process. Mosaic: Piece together 4 images by randomly scaling, randomly cropping, and randomly arranging them. The training data set after data augmentation is used to train the model to improve the generalization of the model. In the training data set, the ratio of the training set, validation set, and test set is 8:1:1.

[0122] The training configuration for the object detection model is: epoch = 150, batch size = 16, the learning rate is set to 0.005, and the SGD learning rate optimizer is used.

[0123] In an implementation of the method for detecting the quality of stamping parts in the production line of the present invention, the step of transporting the stamping part to be detected to the detection position through the production line transmission device further includes:

[0124] The programmable logic controller (PLC) of the production line transmission device generates a signal indicating arrival.

[0125] In an implementation of the method for detecting the quality of stamping parts in the production line of the present invention, the step of collecting the image data 4 of the high-risk points of the defects of the stamping part to be detected further includes:

[0126] After detecting the signal indicating arrival, the image acquisition device collects the image data of multiple high-risk points of the defects of the stamping part to be detected.

[0127] In an implementation of the method for detecting the quality of stamping parts in the production line of the present invention, the step of defect detection on the image data 4 of the high-risk points of the defects of the stamping part to be detected according to the object detection model further includes:

[0128] Use the non-maximum suppression method to delete the detection frames with a large overlap degree, and output an image with the detection frames and defect category information.

[0129] Non-Maximum Suppression (NMS) is a technique in image processing. It is commonly used in object detection, and its main function is to remove redundant detected bounding boxes, only retaining the bounding boxes that are most likely to contain the target object, thus preserving the optimal detection results.

[0130] In object detection, a detector is usually used to detect possible objects and give prediction bounding boxes for their positions and sizes. However, the same object may be detected multiple times, resulting in multiple prediction bounding boxes. At this time, NMS is needed to remove these overlapping bounding boxes and only retain the optimal one.

[0131] Its basic principle is to first find all rectangular regions in the image that may contain the target object and arrange them according to their confidence levels. Then, starting from the rectangle with the highest confidence, traverse all the rectangles. If it is found that the overlapping area of the current rectangle with any previous rectangle is greater than a threshold, the current rectangle is discarded. This makes the number of finally retained prediction bounding boxes the least, while ensuring the accuracy and recall rate of detection. The specific implementation method includes the following steps:

[0132] For each defect category, sort according to the confidence level of the prediction bounding box, and use the prediction bounding box with the highest confidence as the reference;

[0133] Select a bounding box from the remaining prediction bounding boxes that has the largest overlapping area with the reference bounding box. If its overlapping area is greater than a certain threshold, delete it.

[0134] Continue to traverse the remaining prediction bounding boxes until all overlapping areas are less than the threshold or there are no remaining undeleted bounding boxes.

[0135] Then, use the NMS technique to sort the confidence values of each defect category from largest to smallest respectively. Use the prediction bounding box with the highest confidence as the reference, select a bounding box from the remaining prediction bounding boxes that has the largest overlapping area with the reference bounding box. If its overlapping area is greater than the threshold of the intersection over union, delete it, and repeat this step until the final detection result is obtained after excluding all prediction bounding boxes with large overlaps.

[0136] In an implementation of the method for detecting the quality of stamped parts in the production line of the present invention, the method further includes:

[0137] Save the results of defect detection;

[0138] Perform real-time visual display on the results of defect detection;

[0139] Send an alarm when the results of defect detection are abnormal.

[0140] In the above technical solution, the stamping part defects are displayed in a visual manner, the stamping part defects are alarmed in real time, and the stamping part defects can be reviewed in time.

[0141] In one implementation of the stamping part quality detection method of the production line of the present invention, the execution process of the detection method is as follows: Configure an image acquisition device (such as an industrial camera) and a server so that they are in the same local area network as the production line transmission device. When the stamping part to be detected is transmitted to the detection position through the production line transmission device, the programmable logic controller (PLC) of the production line transmission device generates a signal indicating arrival and stores the signal indicating arrival in the form of a Boolean value; Use the snap7 module based on the s7 communication protocol to read the signal indicating arrival and parse it into data that can be processed by the python system program (stored in the server). After the stamping part to be detected arrives at the belt of the production line transmission device, the signal indicating arrival stored in the PLC changes in pulses. The python system program captures this signal indicating arrival in real time and triggers the image acquisition device suspended above the belt to start collecting images of the stamping part to be detected in a multi-process manner. Multi-process collection reduces the waiting time and improves the program operation efficiency. The RGB image obtained by collecting the image of the stamping part to be detected is stored in the server in the form of a dictionary. To further improve the image processing efficiency, use the openCV image processing module to convert the RGB image into a format with a color channel of BGR, and then perform image preprocessing (preprocessing includes filtering and normalization). Input the preprocessed image into the target detection model to start inference. The target detection model scales the input picture size (such as 2448*2048 pixels to 640*640 pixels). After inference, all prediction boxes, confidence levels, and defect categories are obtained. Filter the confidence scores of the prediction boxes below the confidence threshold (such as 0.6) (the maximum value of the product of the confidence levels of different defect categories and the confidence level of the prediction box is used as the confidence score), and filter out the confidence scores below the confidence threshold again to obtain the defect category number and prediction box corresponding to the maximum confidence score. Then use the NMS technology to sort the confidence scores of each defect category from largest to smallest. Take the prediction box with the highest confidence score as the reference, and select a box with the largest overlapping area with the reference box from the remaining prediction boxes. If its overlapping area is greater than the intersection-over-union threshold (the intersection-over-union threshold can be set as needed), then delete it. Repeat this step until all prediction boxes with large overlaps are excluded to obtain the final detection result.

[0142] Use pysides2 to make a on-site display interface for the stamping part defect detection results, and automatically update and display the detection result images with defects.

[0143] In an implementation of the method for detecting the quality of stamped parts on the production line of the present invention, the final detection result is associated with the production information of the stamped parts, such as the stamped part number, production time, etc. If the detection result of the stamped part is abnormal, more data during the production of the stamped part can be obtained through the stamped part number to analyze, trace back, and adjust the configuration of the stamping machine for the cause of the defect.

[0144] To better achieve the object of the present invention, please refer to Figure 5 , the present invention also provides a quality detection device 10 for stamped parts on a production line, which is used to implement any of the above methods, including:

[0145] A target detection model construction module 11, which is used to construct a target detection model based on a neural network through the stamped part defect image;

[0146] A transfer module 12, which is used to transfer the stamped part to be detected to the detection position through the production line transmission equipment;

[0147] An image data acquisition module 13, which is used to acquire the image data 4 of the high-risk points of the defects of the stamped part to be detected;

[0148] A defect detection module 14, which is used to detect the defects of the image data 4 of the high-risk points of the stamped part to be detected according to the target detection model.

[0149] To better achieve the object of the present invention, please refer to Figure 6 , the present invention also provides a quality detection system 100 for stamped parts on a production line, including:

[0150] A stamping device 110, which is used to stamp the production material to generate the stamped part 160 to be detected;

[0151] A transfer device 120, which is used to transfer the stamped part to be detected to the detection area;

[0152] A production line transmission equipment 130, which is arranged in the detection area and is used to transmit the stamped part 160 to be detected to the detection position;

[0153] An image acquisition device 140, which is used to acquire the image data 4 of the high-risk points of the defects of the stamped part 160 to be detected;

[0154] A quality detection unit 150, which is connected to the production line transmission equipment 130 and the image acquisition device 140, and the quality detection unit 150 also includes the quality detection device 10 for stamped parts on the production line as described above.

[0155] Please refer to Figure 7 and Figure 10, in an embodiment of the production line stamping part quality detection system 100 provided by the present invention, when the stamping part to be detected is transported to the detection position by the production line transmission device, the programmable logic controller (PLC) 131 of the production line transmission device generates a signal indicating that the part has arrived. The Python system program stored in the server 151 (the server can be set in the quality detection unit 150 or outside the quality detection unit 150 and connected to the quality detection unit 150) captures this signal in real time, triggering the image acquisition device suspended above the belt to start acquiring images of the stamping part to be detected in a multi-process manner. The RGB images obtained by acquiring the images of the stamping part to be detected are stored in the server in the form of a dictionary. Then, the quality detection unit performs image preprocessing and inputs the preprocessed images into the target detection model to start inference. After inference, if there are defects in the stamping part, defect detection results including prediction boxes, confidence levels, and defect categories are obtained. The visualization unit 180 is connected to the quality detection unit 150 and performs real-time visual display of the stamping part defect detection results (as shown in Figure 10 , for example, there are 4 image acquisition devices numbered 1 to 4. The images acquired by the 1st, 2nd, and 4th image acquisition devices are normal OK after defect detection by the target detection model, while the image acquired by the 3rd image acquisition device is abnormal after defect detection by the target detection model, and the defect detection results including prediction boxes, confidence levels, and defect categories are displayed on the screen). In a further embodiment, when the quality detection unit detects a defect in the stamping part, an alarm unit ( Figure 7 not shown) connected to the quality detection unit 150 issues an alarm. At this time, the staff can stop the production process and can promptly review the stamping part defects and take further troubleshooting and other operations.

[0156] Please refer to Figure 8 and Figure 9 , the production line stamping part quality detection system 100 further includes a support frame 170 for carrying the image acquisition device. The support frame 170 is located above the production line transmission device 130 and includes at least one upper frame 171 and at least one lower frame 172. The upper frame 171 is fixedly arranged above the production line transmission device 130. The upper frame 171 and the lower frame 172 are connected by a first support rod 173. At least one image acquisition device 140 is connected to the upper frame 171 by a second support rod 174. The lower frame 172 has a groove, and a lighting unit 175 is arranged in the groove. The height of the lighting unit 175 from the production line transmission device 130 is flush with the height of the image acquisition device 140 from the production line transmission device 130, which is used to prevent the formation of a shadow of the image acquisition device when the height of the lighting unit 175 is too high during lighting, or to prevent the occlusion of image acquisition when the height of the lighting unit 175 is too low.

[0157] As shown in Figure 11As shown, in an embodiment of the production line stamping part quality inspection system 100 provided by the present invention, the image acquisition device acquires image data of the high-risk defect points of the stamping parts. However, the high-risk defect points of different types of stamping parts may be different. Sometimes, it is necessary to adjust the position of the image acquisition device to completely and clearly acquire the image data of the required high-risk defect points. In this embodiment, a plurality of image acquisition devices are arranged on the support frame, and the position angles of the plurality of image acquisition devices can satisfy the shooting of the high-risk defect points of multiple types of stamping parts. For different types of stamping parts, the image acquisition device corresponding to the position of the high-risk defect point of this type of stamping part is used to acquire the image. Specifically, after the python system program captures the in-place signal of the stamping part in real time, according to the type of the stamping part, it triggers the corresponding image acquisition device that can shoot the high-risk defect point of this type of stamping part to acquire the image, and then the quality inspection unit performs stamping part defect identification. For example Figure 11 10 image acquisition devices are shown in it, which are divided into image acquisition devices numbered 1 to 10 from the upper left to the lower right (the number of this image acquisition device is only for illustration of this implementation scheme. In actual implementation, the number of image acquisition devices can be adjusted according to the actual situation, and the present invention does not limit the number of image acquisition devices). For example, when the stamping part to be detected is the front car door, trigger the 2nd, 4th, 6th, and 8th image acquisition devices to acquire images. When the stamping part to be detected is the fender, trigger the 1st, 5th, 6th, and 10th image acquisition devices to acquire images. In a further implementation manner, for example, when the stamping part to be detected is the front car door, the high-risk defect points of the front car doors of different vehicle models are also different. At this time, the python system program can obtain one or more high-risk defect points of this stamping part according to the stamping part number, stamping part type, and stamping part vehicle type stored in the server, and then trigger the image acquisition device corresponding to the high-risk defect point to acquire images.

[0158] Further embodiments, the lower frame 172 has one or more grooves. For example, when the lower frame 172 has a groove, a plurality of light-emitting units 175 are arranged in the groove. Since the shapes of different types of stampings 160 to be detected are different, when the plurality of light-emitting units 175 in the groove emit light simultaneously for illumination, it may cause local over-illumination and loss of image details due to reflection caused by the shape near the high-risk defect points of the stamping 160 to be detected, resulting in a deterioration of the defect detection effect. At this time, the python system program can control some of the light-emitting units 175 in the groove to illuminate and some of the light-emitting units 175 to turn off the illumination according to the illumination scheme corresponding to the stamping stored in the server 180, so as to make the image of the high-risk defect points of the stamping to be detected clear. The python system program can also analyze and dynamically adjust the opening or closing of some of the light-emitting units 175 among the plurality of light-emitting units 175 in the groove according to parameters such as the image brightness and contrast of the high-risk defect points of the stamping to be detected, so as to collect a clear image of the high-risk defect points of the stamping 160 to be detected.

[0159] It should be known that in the above technical solution, the dynamic opening and closing of some image acquisition devices according to different types of stampings, and the dynamic opening and closing of some light-emitting units according to different types of stampings can be carried out simultaneously or separately.

[0160] Similarly, the technical solution of the above embodiment of the production line stamping quality detection system 100 can also be applied to the production line stamping quality detection method.

[0161] To better achieve the object of the present invention, the present invention also provides a storage medium for storing a computer control program, and the computer control program is used to execute the steps of any one of the above methods.

[0162] The above computer program can be placed in a random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the technical field.

[0163] Compared with the prior art, a production line stamping quality detection method, device and system provided by the present invention build a target recognition model based on a neural network to automatically identify stamping defects instead of manual labor; and optimize the target recognition model to better capture the boundaries of non-rigid targets, further effectively fuse multi-scale features, enhance the ability of feature fusion, and improve the detection effect of stamping defects; display stamping defects in a visual manner, alarm stamping defects in real time and can promptly review stamping defects.

[0164] The above-disclosed content is only a preferred and feasible embodiment of the present invention, and does not limit the scope of the patent application of the present invention. Therefore, all equivalent technical changes made by using the content of the specification and drawings of the present invention fall within the scope of the patent application of the present invention.

Claims

1. A production line stamping parts quality inspection method, characterized in that: The following steps are involved: The neural network-based target detection model is constructed through stamping defect images, including: Construct the backbone network layer, neck layer and head layer; Using a snake convolution module to replace the feature processing module at the last level of the backbone network layer; Using a weighted bidirectional feature pyramid network module to replace all feature concatenation modules of the neck layer; The stamping parts to be inspected are transported to the inspection position through the production line transmission equipment; According to the type of the stamping part to be inspected, the corresponding high-risk defect point is obtained, and the image acquisition device corresponding to the location of the high-risk defect point is controlled to acquire clear image data; wherein the image quality parameters of the image data of the high-risk defect point are analyzed and the opening or closing of the plurality of image acquisition devices corresponding to the location of the high-risk defect point is dynamically adjusted; Defect detection is performed on the image data of the high-risk defect points of the stamping part to be detected according to the target detection model.

2. The method according to claim 1, characterized in that The step of constructing a neural network-based target detection model through stamping part defect images also includes: Annotating the stamping part defect image including crack type and gap type; Data augmentation is performed on the annotated images to construct a training dataset.

3. The method according to claim 2, characterized in that The step of performing data enhancement on the annotated image further comprises: Image enhancements performed on the annotated images include blending, mosaic, and salt and pepper noise.

4. The method according to claim 1, characterized in that: The step of transferring the stamping part to be inspected to the inspection position through the production line transmission equipment also includes: The programmable logic controller of the production line conveyor generates an arrival signal.

5. The method according to claim 4, characterized in that The step of collecting image data of high-risk points of defects in the stamping part to be detected further includes: After detecting the arrival signal, the image acquisition device collects image data of multiple high-risk defect points of the stamping part to be inspected.

6. The method according to claim 1, characterized in that The step of performing defect detection on the image data of the high-risk defect points of the stamping part to be detected according to the target detection model further includes: The non-maximum suppression method is used to delete the detection frames with large overlap, and an image with detection frames and defect category information is output.

7. The method according to claim 1, characterized in that The method further comprises: Saving the defect detection result; Performing real-time visual display of the defect detection results; When the result of the defect detection is abnormal, an alarm is issued.

8. A production line stamping parts quality inspection device, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The target detection model building module is used to build a neural network-based target detection model through stamping defect images; A conveying module is used to convey the stamping parts to be inspected to the inspection position through the production line transmission equipment; An image data acquisition module, used to acquire image data of high-risk points of defects in the stamping part to be detected; The defect detection module is used to perform defect detection on the image data of the high-risk defect points of the stamping part to be detected according to the target detection model.

9. A production line stamping parts quality inspection system, comprising: Stamping equipment, used to stamp production materials to generate stamped parts to be tested; A transfer device, used for transferring the stamping part to be inspected to the inspection area; Production line transmission equipment, arranged in the detection area, for transmitting the stamping parts to be detected to the detection position; An image acquisition device is used to acquire image data of high-risk points of defects in the stamping part to be inspected; A quality inspection unit is connected to the production line transmission device and the image acquisition device, and is characterized in that the quality inspection unit also includes the production line stamping part quality inspection device as described in claim 8.

10. A storage medium for storing a computer control program, characterized in that: The computer control program is used to execute the method according to any one of claims 1 to 7.

Images