Production line stamping part quality detection method, device and system

Abstract

The invention provides a production line stamping part quality detection method and device. The method comprises the following steps: constructing a target detection model based on a neural network through a stamping part defect image; the stamping part to be detected is conveyed to a detection position through production line conveying equipment; collecting image data of defect high-risk points of the stamping part to be detected; and performing defect detection on the image data of the defect high-risk points of the to-be-detected stamping part according to the target detection model. The invention further provides a production line stamping part quality detection system.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to the field of automated stamping part defect detection. Background Art

[0002] At present, the online quality inspection of stamping parts in production lines mainly relies on manual inspection. Due to the cycle time limit of stamping parts production, it is impossible to achieve 100% inspection. The defects of stamping parts are mainly cracking, thinning, wrinkling, etc. Manual inspection is inefficient and prone to missed inspections, resulting in the risk of defective products flowing out, so this problem needs to be solved. Summary of the invention

[0003] In order to solve the technical problem of low inspection efficiency and easy missed inspection in stamping parts production, the present invention proposes a production line stamping parts quality inspection method, comprising the following steps:

[0004] Construct a neural network-based target detection model through stamping defect images;

[0005] The stamping parts to be inspected are transported to the inspection position through the production line transmission equipment;

[0006] Collecting image data of high-risk points of defects in the stamping part to be inspected;

[0007] Defect detection is performed on the image data of the high-risk defect points of the stamping part to be detected according to the target detection model.

[0008] In the above-mentioned production line stamping parts quality inspection method, the step of constructing a neural network-based target detection model through stamping parts defect images further includes:

[0009] Construct the backbone network layer, neck layer and head layer;

[0010] Using a snake convolution module to replace the feature processing module at the last level of the backbone network layer;

[0011] A weighted bidirectional feature pyramid network module is used to replace all feature concatenation modules in the neck layer.

[0012] In the above technical solution, the snake convolution can better capture the non-rigid target boundaries, the weighted bidirectional feature pyramid network can further effectively fuse multi-scale features, and by introducing bidirectional connections, the feature fusion capability is enhanced, thereby improving the detection effect of stamping defects.

[0013] In the above-mentioned production line stamping parts quality inspection method, the step of constructing a neural network-based target detection model through stamping parts defect images also includes:

[0014] Annotating the stamping part defect image including crack type and gap type;

[0015] Data augmentation is performed on the annotated images to construct a training dataset.

[0016] In the above technical solution, cracking of stamping parts can be accurately identified by marking the crack type and the crack type of the defect image.

[0017] In the above-mentioned production line stamping parts quality inspection method, the step of performing data enhancement on the annotated image further includes:

[0018] Image enhancements performed on the annotated images include blending, mosaic, and salt and pepper noise.

[0019] In the above-mentioned production line stamping parts quality inspection method, the step of transmitting the stamping parts to be inspected to the inspection position through the production line transmission equipment also includes:

[0020] The programmable logic controller of the production line conveyor generates an arrival signal.

[0021] In the above-mentioned production line stamping parts quality inspection method, the step of collecting image data of high-risk points of defects in the stamping parts to be inspected further includes:

[0022] After detecting the arrival signal, the image acquisition device collects image data of multiple high-risk defect points of the stamping part to be inspected.

[0023] In the above-mentioned production line stamping parts quality inspection method, the step of performing defect detection on the image data of the high risk points of defects of the stamping parts to be inspected according to the target detection model further includes:

[0024] The non-maximum suppression method is used to delete the detection frames with large overlap, and an image with detection frames and defect category information is output.

[0025] In the above-mentioned production line stamping parts quality inspection method, the method further includes:

[0026] Saving the defect detection result;

[0027] Performing real-time visual display of the defect detection results;

[0028] When the result of the defect detection is abnormal, an alarm is issued.

[0029] In the above technical solution, the defects of stamping parts are displayed in a visual manner, and the defects of stamping parts are alarmed in real time and the defects of stamping parts can be reviewed in time.

[0030] In order to better achieve the purpose of the present invention, the present invention also provides a production line stamping parts quality detection device, which is used to implement any of the above methods, including:

[0031] The target detection model building module is used to build a neural network-based target detection model through stamping defect images;

[0032] A conveying module is used to convey the stamping parts to be inspected to the inspection position through the production line transmission equipment;

[0033] An image data acquisition module, used to acquire image data of high-risk points of defects in the stamping part to be detected;

[0034] The defect detection module is used to perform defect detection on the image data of the high-risk defect points of the stamping part to be detected according to the target detection model.

[0035] In order to better achieve the purpose of the present invention, the present invention also provides a production line stamping parts quality inspection system, comprising:

[0036] Stamping equipment, used to stamp production materials to generate stamped parts to be tested;

[0037] A transfer device, used for transferring the stamping part to be inspected to the inspection area;

[0038] Production line transmission equipment, arranged in the detection area, for transmitting the stamping parts to be detected to the detection position;

[0039] An image acquisition device is used to acquire image data of high-risk points of defects in the stamping part to be inspected;

[0040] A quality inspection unit is connected to the production line transmission device and the image acquisition device, and the quality inspection unit also includes the production line stamping parts quality inspection device as mentioned above.

[0041] In order to better achieve the purpose of the present invention, the present invention also provides a storage medium for storing a computer control program, wherein the computer control program is used to execute the steps of any of the above methods.

[0042] Compared with the prior art, the present invention provides a production line stamping parts quality inspection method, device and system, which replaces manual automatic identification of stamping parts defects by building a target recognition model based on a neural network; and optimizes the target recognition model to better capture non-rigid target boundaries, further effectively integrate multi-scale features, enhance the feature fusion capability, and improve the detection effect of stamping parts defects; display stamping parts defects in a visual manner, alarm stamping parts defects in real time, and can promptly review stamping parts defects.

[0043] To further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are only for reference and description and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flow chart of a method for quality inspection of stamping parts on a production line according to an embodiment of the present invention is shown.

[0045] Figure 2 A diagram showing the architecture of an object detection model in the prior art.

[0046] Figure 3 A diagram illustrating the architecture of an optimized object detection model in one embodiment of the present invention.

[0047] Figure 4 A schematic diagram of feature fusion of a weighted bidirectional feature pyramid network module in one embodiment of the present invention is shown.

[0048] Figure 5 A block diagram of a production line stamping parts quality inspection device according to an embodiment of the present invention is shown.

[0049] Figure 6 A block diagram of a production line stamping parts quality inspection system according to an embodiment of the present invention is shown.

[0050] Figure 7 A block diagram of a production line stamping parts quality inspection system according to another embodiment of the present invention is shown.

[0051] Figure 8 A top view of defect identification of a stamping part to be inspected on a production line transmission device in one embodiment of the present invention is shown.

[0052] Fig. 9 A front view of defect identification of a stamping part to be inspected on a production line transmission device in one embodiment of the present invention is shown.

[0053] Fig.10 A schematic diagram is shown of real-time visual display of defect detection results of a stamping part to be inspected in one embodiment of the present invention.

[0054] Fig.11 A schematic diagram is shown of multiple image acquisition devices acquiring images of different types of stamping parts in one embodiment of the present invention.

[0055] Wherein, the reference numerals are:

[0056] S1-S4... Steps

[0057] 1…Backbone network layer

[0058] 2…Neck layer

[0059] 3…First layer

[0060] 4…Image data of high-risk spots of defects in stamping parts to be inspected

[0061] 5…Snake Convolution Module

[0062] 6…Weighted Bidirectional Feature Pyramid Network Module

[0063] 10…Production line stamping parts quality inspection device

[0064] 11…Object Detection Model Building Module

[0065] 12…Transmission module

[0066] 13…Image data acquisition module

[0067] 14…Defect Detection Module

[0068] 100…Production Line Stamping Parts Quality Inspection System

[0069] 110…Stamping equipment

[0070] 120…Transfer device

[0071] 130…Production line transmission equipment

[0072] 131…Programmable logic controller

[0073] 140…Image acquisition equipment

[0074] 150…Quality Inspection Unit

[0075] 151…Server

[0076] 160…Stamping parts to be tested

[0077] 170…Support frame

[0078] 171…Upper frame

[0079] 172…lower frame

[0080] 173…First support rod

[0081] 174…Second support rod

[0082] 175...Light-emitting unit

[0083] 180...Visualization unit DETAILED DESCRIPTION

[0084] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments to further understand the purpose, solution and beneficial technical effects of the present invention. Obviously, the specific embodiments described in the present invention are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments disclosed in the present invention, all other embodiments obtained by ordinary technicians in this field without spending creative labor also belong to the scope disclosed by the technical solution of the present invention.

[0085] It should be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0086] Certain words are used in the specification and the appended claims to refer to specific components or parts. It should be understood by those skilled in the art that technical users or manufacturers may refer to the same component or part by different nouns or terms. This specification and the appended claims do not distinguish components or parts by differences in names, but by differences in functions.

[0087] In the present invention, the directions or positional relationships indicated by the terms "upper", "lower", "left", "right", "front", "back", "top", "bottom", "inner", "outer", "middle", "vertical", "horizontal", "lateral", "longitudinal" and the like are based on the directions or positional relationships shown in the drawings. These terms are mainly used to better describe the present invention and its embodiments, and are not used to limit the indicated devices, elements or components to have a specific direction, or to be constructed and operated in a specific direction.

[0088] In addition, the terms "installed", "set", "provided with", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection, or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be an internal connection between two devices, elements, or components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0089] To better understand the technical solution of the present invention, please refer to Figure 1 In order to solve the technical problems of low inspection efficiency and easy missed inspection in stamping parts production, the present invention proposes a production line stamping parts quality inspection method, comprising the following steps:

[0090] Step S1: constructing a neural network-based target detection model through stamping part defect images;

[0091] Step S2: The stamped parts to be inspected are transported to the inspection position through the production line transmission equipment;

[0092] Step S3: collecting image data 4 of high-risk points of defects in the stamping part to be inspected;

[0093] Step S4: performing defect detection on the image data 4 of the high-risk points of defects in the stamping part to be detected according to the target detection model.

[0094] In one implementation of the production line stamping parts quality inspection method of the present invention, the step of constructing a neural network-based target detection model through stamping parts defect images further includes:

[0095] Construct the backbone network layer (backbone) 1, the neck layer (neck) 2 and the head layer (head) 3;

[0096] Use the snake convolution module (DSConv)5 to replace the feature processing module at the last level of backbone network layer 1;

[0097] Use the weighted bidirectional feature pyramid network module (BiFPN) 6 to replace all feature concatenation (Concat) modules in the neck layer 2;

[0098] The loss functions used by the target detection model are divided into prediction box loss, target confidence loss and defect category loss. Among them, the prediction box loss function is EIOU Loss, and the target confidence loss and defect category loss functions are binary cross entropy function BCEWithLogitsLoss with sigmoid.

[0099] Specifically, in order to better understand the technical solution of the present invention, please refer to Figure 2 and Figure 3 , Figure 2 The architecture diagram of the target detection model in the prior art is shown. Figure 3The architecture diagram of the optimized target detection model in one embodiment of the present invention is shown. When constructing a target detection model based on a neural network, any one of the yolov5 series target detection models can be selected. The yolov5 series target detection model backbone network layer 1, neck layer 2 and head layer 3 can perform target recognition on the image data of the input defect high risk point, and the constituent modules of each layer are all existing technologies, which will not be described in detail here.

[0100] In order to better detect the defects of stamping parts, the target detection model is optimized, and the feature processing module (C3 module) at the last level of the backbone network layer 1 is replaced by the snake convolution module 5;

[0101] For the principle of snake convolution:

[0102] Given a standard 2D convolution coordinate K, the center coordinate is K i =(x i +y i ), a 3×3 convolution kernel K is expressed as:

[0103] K={(x-1,y-1),(x-1,y),…,(x+1,y+1)} (1)

[0104] In order to give the convolution kernel more flexibility and enable it to focus on the complex geometric features of the target, a deformation offset Δ is introduced. Dynamic snake convolution linearizes the standard convolution kernel in both the x-axis and y-axis directions. Consider a convolution kernel of size 9. Taking the x-axis direction as an example, the specific position of each grid in K is expressed as: K i±c =(x i±c +y i±c ), where c = 0, 1, 2, 3, 4 represents the horizontal distance from the center grid. Each grid position K in the convolution kernel K i±c The selection of is a cumulative process. From the center position K i Initially, the position of the grid away from the center depends on the position of the previous grid: K i±1 Relative to K i The offset Δ={δ|δ∈[-1,1]} is added. Therefore, the offset needs to be accumulated Σ to ensure that the convolution kernel conforms to the linear morphology structure. The change in the x-axis direction is:

[0105]

[0106] The change in the y-axis direction is:

[0107]

[0108] Since the offset Δ is usually a decimal, and the coordinates are usually in integer form, a double-line interpolation value is used, expressed as:

[0109] K=∑ K′ B(K′,K)·K′ (4)

[0110] Where K represents the decimal position of equation (2) and equation (3), K′ lists all integer spatial positions, and B is the bilinear interpolation kernel, which can be decomposed into two one-dimensional kernels, namely:

[0111] B(K′,K)=b(K x ,K′ x )·b(K y ,K′ y ) (5)

[0112] Use the weighted bidirectional feature pyramid network module to replace all feature concatenation (Concat) modules in the neck layer 2. BiFPN is an improved feature pyramid network (FPN) structure. Through bidirectional connections and adaptive feature adjustment, it improves the performance of object detection and semantic segmentation tasks, enables the neural network to better understand and interpret multi-scale information, and improves the accuracy of small target detection. Figure 4 As shown in the figure, the introduction of BiFPN in yolov5 can more effectively detect small target objects on stamping parts. Small targets usually occupy fewer pixels in the image and are therefore more easily ignored or misjudged.

[0113] The introduction of BiFPN allows information to propagate bidirectionally between different resolution levels, thereby better fusing multi-scale information. This helps the model understand targets of different sizes more comprehensively and improves the detection performance of multi-scale objects. At the same time, it also improves the contextual understanding of objects and helps reduce false positives or negatives.

[0114] In the above technical solution, the snake convolution can better capture the non-rigid target boundaries, the weighted bidirectional feature pyramid network can further effectively fuse multi-scale features, and by introducing bidirectional connections, the feature fusion capability is enhanced, thereby improving the detection effect of stamping defects.

[0115] In one implementation of the stamping parts quality inspection method of the production line of the present invention, the step of constructing a neural network-based target detection model through the stamping parts defect image also includes:

[0116] Label the stamping parts defect images including crack types and split types;

[0117] Data augmentation is performed on the annotated images to construct a training dataset.

[0118] In the above technical solution, cracking of stamping parts can be accurately identified by marking the crack type and the crack type of the defect image.

[0119] In one implementation of the production line stamping parts quality inspection method of the present invention, the step of performing data enhancement on the annotated image further includes:

[0120] Image enhancements performed on the annotated images include blending, mosaic, and salt and pepper noise.

[0121] Specifically, labelimg is used to annotate the defect images of stamping parts, and the annotated defect categories are divided into at least two categories: cracks and fissures. Then, one or more of the mixing (mixup), mosaic (mosaic) and salt and pepper noise methods are used for data enhancement to produce a training data set. Mixing: Randomly select two images from each batch and mix them in a certain proportion to generate a new image. It should be noted that the entire training process only uses mixed new images for training, and the original images do not participate in the training process. Mosaic: 4 pictures are spliced ​​by random scaling, random cropping, and random arrangement. The model is trained with the data-enhanced training data set to improve the generalization of the model. In the training data set, the ratio of training set, validation set, and test set is 8:1:1.

[0122] The training configuration for the object detection model is: epoch = 150, batch size = 16, the learning rate is set to 0.005, and the SGD learning rate optimizer is used.

[0123] In one implementation of the production line stamping parts quality inspection method of the present invention, the step of transmitting the stamping parts to be inspected to the inspection position through the production line transmission equipment also includes:

[0124] The programmable logic controller (PLC) of the production line conveyor generates an arrival signal.

[0125] In one implementation of the production line stamping parts quality inspection method of the present invention, the step of collecting image data 4 of high risk points of defects in the stamping parts to be inspected further includes:

[0126] After detecting the arrival signal, the image acquisition device collects image data of multiple high-risk defect points of the stamping part to be inspected.

[0127] In one implementation of the production line stamping parts quality inspection method of the present invention, the step of performing defect detection on the image data 4 of the high risk points of the stamping parts to be inspected according to the target detection model further includes:

[0128] The non-maximum suppression method is used to delete the detection frames with large overlap, and an image with detection frames and defect category information is output.

[0129] Non-Maximum Suppression (NMS) is a technique in image processing. It is usually used in target detection. Its main function is to remove redundant detected frames and only retain the frames that are most likely to contain the target object, thus retaining the best detection results.

[0130] In object detection, a detector is usually used to detect possible objects and give a prediction box of their position and size. However, the same object may be detected multiple times, resulting in multiple prediction boxes. In this case, NMS is needed to remove these overlapping boxes and keep only the best one.

[0131] The basic principle is to first find all the rectangular areas that may contain the target object in the image and arrange them according to their confidence. Then, starting from the rectangle with the highest confidence, traverse all rectangles. If the overlapping area between the current rectangle and any previous rectangle is greater than a threshold, the current rectangle is discarded. This ensures that the number of prediction boxes retained in the end is minimal, while at the same time ensuring the accuracy and recall rate of detection. The specific implementation method includes the following steps:

[0132] For each defect category, the prediction boxes are sorted according to their confidence, and the prediction box with the highest confidence is used as the benchmark;

[0133] From the remaining predicted boxes, a box with the largest overlap area with the reference box is selected and deleted if its overlap area is larger than a certain threshold.

[0134] Continue to traverse the remaining prediction boxes until all overlapping areas are smaller than the threshold, or no deleted boxes remain.

[0135] Then, the NMS technology is used to sort the confidence values ​​of each defect category from large to small, and the prediction box with the highest confidence is used as the benchmark. From the remaining prediction boxes, a box with the largest overlapping area with the benchmark box is selected. If its overlapping area is greater than the intersection-over-union ratio threshold, it is deleted. This step is repeated until all prediction boxes with large overlaps are excluded to obtain the final detection result.

[0136] In one implementation of the method for quality inspection of stamping parts in a production line of the present invention, the method further comprises:

[0137] Save the results of defect detection;

[0138] Real-time visualization of defect detection results;

[0139] An alarm is issued when the defect detection results are abnormal.

[0140] In the above technical solution, the defects of stamping parts are displayed in a visual manner, and the defects of stamping parts are alarmed in real time and the defects of stamping parts can be reviewed in time.

[0141] In one implementation of the production line stamping parts quality inspection method of the present invention, the execution process of the inspection method is to configure the image acquisition device (such as an industrial camera) and the server so that they are in the same local area network as the production line transmission device. When the stamping part to be inspected is transmitted to the inspection position through the production line transmission device, the programmable logic controller (PLC) of the production line transmission device generates an in-place signal and stores the in-place signal in the form of a Boolean value; the snap7 module based on the s7 communication protocol is used to read the in-place signal and parse it into data that can be processed by the python system program (stored in the server). After the stamping part to be inspected is in place on the belt of the production line transmission device, the in-place signal stored in the PLC undergoes a pulse change, and the python system program captures this in-place signal in real time, triggering the image acquisition device suspended above the belt to start collecting images of the stamping part to be inspected in a multi-process manner. Multi-process collection reduces waiting time and improves program operation efficiency. The RGB images obtained by collecting the images of the stamping parts to be inspected are stored in the server in the form of a dictionary. In order to further improve the efficiency of image processing, the openCV image processing module is used to convert the RGB images into BGR format with color channels, and then image preprocessing (preprocessing includes filtering and homogenization) is performed. The preprocessed images are input into the target detection model to start reasoning. The target detection model scales the input image size (for example, 2448*2048 pixels to 640*640 pixels. After reasoning, all prediction boxes, confidence levels and defect categories are obtained, and the prediction box confidence scores below the confidence threshold (for example, 0.6) are filtered (not included). The maximum value of the product of the confidence of the same defect category and the confidence of the prediction box is taken as the confidence score), and the confidence scores below the confidence threshold are filtered out again to obtain the defect category number and prediction box corresponding to the maximum confidence score. Then, the confidence scores of each defect category are sorted from large to small using the NMS technology, and the prediction box with the highest confidence score is used as the benchmark. From the remaining prediction boxes, a box with the largest overlap area with the benchmark box is selected. If its overlapping area is greater than the threshold of the intersection-over-union ratio (the threshold of the intersection-over-union ratio can be set as needed), it is deleted, and this step is repeated until all prediction boxes with large overlaps are excluded to obtain the final detection result.

[0142] Use pysides2 to create an on-site display interface for stamping part defect detection results, and automatically update and display the detection result image with defects.

[0143] In one implementation of the production line stamping parts quality inspection method of the present invention, the final inspection result is associated with the production information of the stamping parts, such as the stamping part number, production time, etc. If the stamping part inspection result is abnormal, more data on the stamping part production can be obtained through the stamping part number to analyze the cause of the defect, trace back, and adjust the stamping machine configuration.

[0144] In order to better achieve the purpose of the present invention, please refer to Figure 5 The present invention also provides a production line stamping parts quality inspection device 10, which is used to implement any of the above methods, including:

[0145] The target detection model building module 11 is used to build a target detection model based on a neural network through the stamping part defect image;

[0146] The conveying module 12 is used to convey the stamping parts to be inspected to the inspection position through the production line transmission equipment;

[0147] An image data acquisition module 13, used to acquire image data 4 of high-risk points of defects in stamping parts to be detected;

[0148] The defect detection module 14 is used to perform defect detection on the image data 4 of high-risk points of defects in the stamping part to be detected according to the target detection model.

[0149] In order to better achieve the purpose of the present invention, please refer to Figure 6 The present invention also provides a production line stamping parts quality inspection system 100, comprising:

[0150] A stamping device 110, used for stamping production materials to generate stamped parts 160 to be inspected;

[0151] A transfer device 120, used to transfer the stamping parts to be inspected to the inspection area;

[0152] The production line transmission device 130 is arranged in the detection area and is used to transmit the stamping part 160 to be detected to the detection position;

[0153] An image acquisition device 140 is used to acquire image data 4 of a high risk point of a defect of a stamping part 160 to be inspected;

[0154] The quality inspection unit 150 is connected to the production line transmission device 130 and the image acquisition device 140 . The quality inspection unit 150 also includes the production line stamping part quality inspection device 10 as described above.

[0155] See also Figure 7 and Fig.10In an embodiment of the production line stamping parts quality inspection system 100 provided by the present invention, when the stamping parts to be inspected are transmitted to the inspection position through the production line transmission equipment, the programmable logic controller (PLC) 131 of the production line transmission equipment generates an in-place signal, which is stored in the server 151 (the server can be set in the quality inspection unit 150, or it can be set outside the quality inspection unit 150 and connected to the quality inspection unit 150). The python system program in the real-time captures this in-place signal, triggering the image acquisition device suspended above the belt to start collecting images of the stamping parts to be inspected in a multi-process manner. The RGB images obtained by collecting the images of the stamping parts to be inspected are stored in the server in the form of a dictionary, and then the quality inspection unit performs image preprocessing, and inputs the preprocessed images into the target detection model to start reasoning. After reasoning, if there are defects in the stamping parts, the defect detection results of the prediction box, confidence and defect category are obtained. The visualization unit 180 is connected to the quality inspection unit 150, and performs real-time visual display of the stamping parts defect detection results (such as Fig.10 As shown in, for example, there are 4 image acquisition devices No. 1 to No. 4, among which the images acquired by the image acquisition devices No. 1, 2 and 4 are normal after the defect detection by the target detection model, while the image acquired by the image acquisition device No. 3 is abnormal after the defect detection by the target detection model, and the defect detection result including the prediction box, confidence and defect category is displayed on the screen). In a further embodiment, when the quality inspection unit detects that the stamping part has a defect, the alarm unit ( Figure 7 The operator may stop the production process and promptly review the stamping defects and take further investigations.

[0156] See also Figure 8 and Fig. 9 The production line stamping parts quality inspection system 100 also includes a support frame 170 that carries the image acquisition device. The support frame 170 is located above the production line transmission device 130, and includes at least one upper frame 171 and at least one lower frame 172. The upper frame 171 is fixedly arranged above the production line transmission device 130. The upper frame 171 and the lower frame 172 are connected by a first support rod 173. At least one image acquisition device 140 is connected to the upper frame 171 through a second support rod 174. The lower frame 172 has a groove, and a light-emitting unit 175 is provided in the groove. The height of the light-emitting unit 175 from the production line transmission device 130 is flush with the height of the image acquisition device 140 from the production line transmission device 130, which is used to prevent the shadow of the image acquisition device from being formed when the light-emitting unit 175 is too high for lighting, or to prevent the image acquisition from being blocked when the light-emitting unit 175 is too low.

[0157] like Fig.11As shown, in an embodiment of the production line stamping parts quality inspection system 100 provided by the present invention, the image acquisition device collects image data of the high-risk defect points of the stamping parts, and the high-risk defect points of different types of stamping parts may be different. Sometimes it is necessary to adjust the position of the image acquisition device to be able to completely and clearly collect the image data of the high-risk defect points. In this embodiment, a plurality of image acquisition devices are arranged on the support frame, and the position angles of the plurality of image acquisition devices can meet the requirements of photographing the high-risk defect points of multiple types of stamping parts. For different types of stamping parts, images are collected by the image acquisition devices at the corresponding positions of the high-risk defect points of the stamping parts of that type. Specifically, after the python system program captures the stamping parts' arrival signals in real time, according to the type of stamping parts, the corresponding image acquisition devices that can photograph the high-risk defect points of the stamping parts of that type are triggered to collect images, and then the quality inspection unit performs stamping parts defect identification. For example Fig.11 10 image acquisition devices are shown in the figure, which are numbered 1 to 10 from the upper left to the lower right (the number of image acquisition devices is only a schematic diagram of this embodiment. The number of image acquisition devices can be adjusted according to the actual situation during actual implementation. The present invention does not limit the number of image acquisition devices). For example, when the stamped part to be detected is the front door, image acquisition devices 2, 4, 6 and 8 are triggered to collect images. When the stamped part to be detected is the fender, image acquisition devices 1, 5, 6 and 10 are triggered to collect images. In a further embodiment, for example, when the stamped part to be detected is the front door, the high-risk defect points of the front doors of different models are also different. At this time, the python system program can obtain one or more high-risk defect points of the stamped part according to the stamped part number, stamped part type, and stamped part vehicle type stored in the server, and then trigger the image acquisition device corresponding to the high-risk defect point to collect images.

[0158] In a further embodiment, the lower frame 172 has one or more grooves. For example, when the lower frame 172 has a groove, a plurality of light-emitting units 175 are arranged in the groove. Since different types of stamping parts 160 to be inspected have different shapes, when the plurality of light-emitting units 175 in the groove emit light for illumination at the same time, the shape near the high-risk defect point of the stamping part 160 to be inspected may produce reflections, resulting in local overbrightness and loss of image details, resulting in poor defect detection effect. At this time, the python system program can control some of the light-emitting units 175 in the groove to illuminate according to the lighting scheme corresponding to the stamping part stored in the server 180, and some of the light-emitting units 175 turn off the lighting, so that the collected image of the high-risk defect point of the stamping part to be inspected is clear. The python system program can also dynamically adjust the opening or closing of some of the light-emitting units 175 in the groove according to the image brightness, contrast and other parameters of the high-risk defect point of the stamping part to be inspected in real time to collect a clear image of the high-risk defect point of the stamping part 160 to be inspected.

[0159] It should be known that in the above technical solution, the dynamic opening and closing of some image acquisition devices according to different types of stamping parts, and the dynamic opening and closing of some light-emitting units according to different types of stamping parts can be performed simultaneously or separately.

[0160] Likewise, the technical solution of the above-mentioned production line stamping parts quality inspection system 100 embodiment can also be applied to the production line stamping parts quality inspection method.

[0161] In order to better achieve the purpose of the present invention, the present invention also provides a storage medium for storing a computer control program, and the computer control program is used to execute the steps of any of the above methods.

[0162] The above-mentioned computer program can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the technical field.

[0163] Compared with the prior art, the present invention provides a production line stamping parts quality inspection method, device and system, which replaces manual automatic identification of stamping parts defects by building a target recognition model based on a neural network; and optimizes the target recognition model to better capture non-rigid target boundaries, further effectively integrate multi-scale features, enhance the feature fusion capability, and improve the detection effect of stamping parts defects; display stamping parts defects in a visual manner, alarm stamping parts defects in real time, and can promptly review stamping parts defects.

[0164] The above disclosed contents are only preferred feasible embodiments of the present invention, and do not limit the scope of the patent application of the present invention. Therefore, all equivalent technical changes made using the contents of the specification and drawings of the present invention fall within the scope of the patent application of the present invention.

Claims

1. A production line stamping parts quality inspection method, characterized in that: The following steps are involved: Construct a neural network-based target detection model through stamping defect images; The stamping parts to be inspected are transported to the inspection position through the production line transmission equipment; Collecting image data of high-risk points of defects in the stamping part to be inspected; Defect detection is performed on the image data of the high-risk defect points of the stamping part to be detected according to the target detection model.

2. The method according to claim 1, characterized in that The step of constructing a neural network-based target detection model through stamping part defect images further includes: Construct the backbone network layer, neck layer and head layer; Using a snake convolution module to replace the feature processing module at the last level of the backbone network layer; A weighted bidirectional feature pyramid network module is used to replace all feature concatenation modules in the neck layer.

3. The method according to claim 2, characterized in that The step of constructing a neural network-based target detection model through stamping part defect images also includes: Annotating the stamping part defect image including crack type and gap type; Data augmentation is performed on the annotated images to construct a training dataset.

4. The method according to claim 3, characterized in that The step of performing data enhancement on the annotated image further comprises: Image enhancements performed on the annotated images include blending, mosaic, and salt and pepper noise.

5. The method according to claim 1, characterized in that The step of transferring the stamping part to be inspected to the inspection position through the production line transmission equipment also includes: The programmable logic controller of the production line conveyor generates an arrival signal.

6. The method according to claim 5, characterized in that The step of collecting image data of high-risk points of defects in the stamping part to be detected further includes: After detecting the arrival signal, the image acquisition device collects image data of multiple high-risk defect points of the stamping part to be inspected.

7. The method according to claim 1, characterized in that The step of performing defect detection on the image data of the high-risk defect points of the stamping part to be detected according to the target detection model further includes: The non-maximum suppression method is used to delete the detection frames with large overlap, and an image with detection frames and defect category information is output.

8. The method according to claim 1, characterized in that The method further comprises: Saving the defect detection result; Performing real-time visual display of the defect detection results; When the result of the defect detection is abnormal, an alarm is issued.

9. A production line stamping parts quality inspection device, used to implement the method according to any one of claims 1 to 8, characterized in that: include: The target detection model building module is used to build a neural network-based target detection model through stamping defect images; A conveying module is used to convey the stamping parts to be inspected to the inspection position through the production line transmission equipment; An image data acquisition module, used to acquire image data of high-risk points of defects in the stamping part to be detected; The defect detection module is used to perform defect detection on the image data of the high-risk defect points of the stamping part to be detected according to the target detection model.

10. A production line stamping parts quality inspection system, comprising: Stamping equipment, used to stamp production materials to generate stamped parts to be tested; A transfer device, used for transferring the stamping part to be inspected to the inspection area; Production line transmission equipment, arranged in the detection area, for transmitting the stamping parts to be detected to the detection position; An image acquisition device is used to acquire image data of high-risk points of defects in the stamping part to be inspected; A quality inspection unit is connected to the production line transmission device and the image acquisition device, and is characterized in that the quality inspection unit also includes the production line stamping part quality inspection device as described in claim 9.

11. A storage medium for storing a computer control program, characterized in that: The computer control program is used to execute the method according to any one of claims 1 to 8.

Images