Defect detection model training and defect detection method, device, equipment and medium

Abstract

The present application relates to the technical field of automatic detection, and discloses a defect detection model training method, a defect detection method, a device, equipment and a medium, the defect detection model training method comprising: acquiring a first image corresponding to a defect area and a second image corresponding to a non-defect area of a plurality of first stamping sheet metal parts with surface defects to form an image pair; performing image feature extraction on the first image and the second image in each image pair respectively to obtain first image feature data corresponding to the first image and second image feature data corresponding to the second image; removing the image features in the first image feature data that are the same as the second image feature data and the defect label of the corresponding first image to construct a training set; training an initial defect detection model to obtain a target defect detection model. The present application avoids false detection of images caused by part surface structure of the trained detection model, reduces the false detection rate, and can realize accurate detection of surface defects of stamping sheet metal parts.

Description

Technical Field

[0001] This invention relates to the field of automatic detection technology, specifically to defect detection model training and defect detection methods, devices, equipment and media. Background Technology

[0002] As a benchmark of high-end manufacturing, the automotive industry places great emphasis on quality control during the manufacturing process. The stamping process, as the first step in automobile production, is the first hurdle in determining the styling and appearance quality of a car; any defect in the appearance of a part will affect the overall quality of the vehicle. Therefore, strict control over the appearance quality of parts is essential during the stamping production process.

[0003] Common defects in the stamping process include pressure scratches, dents, cracks, and hidden defects. Currently, most automotive OEMs use online and offline manual inspection methods for quality control. The appearance quality of parts is judged by visual observation and touch; however, the effectiveness of manual inspection is affected by factors such as work time and intensity. In recent years, with the rapid development of machine vision technology, especially in the field of industrial defect detection, the technology of defect identification using deep learning algorithms such as object detection and change detection has become increasingly mature, significantly improving detection capabilities.

[0004] However, since stamped sheet metal parts often have irregular shapes and sharp edges, the image features of such structures are similar to cracks. Detection models that use traditional deep learning algorithms for defect identification often mistake these structures for cracks, resulting in a large number of false detections. Summary of the Invention

[0005] In view of this, the present invention provides a defect detection model training and defect detection method, apparatus, equipment and medium to solve the problems of high false detection rate and poor accuracy of detection results when traditional deep learning algorithms are used to identify defects in stamped sheet metal parts.

[0006] In a first aspect, the present invention provides a defect detection model training method, the method comprising:

[0007] Acquire a first image corresponding to the defect area of ​​a number of first stamped sheet metal parts with surface defects, and a second image of the defect area corresponding to the defect-free area on the surface of a standard first stamped sheet metal part. The first image is marked with defect labels.

[0008] Based on the defect region, establish the association between the first image and the second image to form an image pair;

[0009] Image features are extracted from the first image and the second image in each image pair to obtain the first image feature data corresponding to the first image and the second image feature data corresponding to the second image.

[0010] Remove image features from the first image feature data that are identical to those in the second image feature data to obtain the third image feature data;

[0011] A training set is constructed based on the feature data of each third image and the defect annotations of the corresponding first image;

[0012] The initial defect detection model is trained using the training set to obtain the target defect detection model corresponding to the first stamped sheet metal part.

[0013] This invention constructs image pairs by combining images of defective areas on the surface of stamped sheet metal parts with corresponding images of defect-free standard stamped sheet metal parts. Simultaneous image feature extraction is then performed. The image features corresponding to the defective areas are then discarded to remove any overlap with the features of the defect-free areas. This ensures that the retained image features are only related to defects on the surface of the stamped sheet metal parts. These retained image features are then used as a training set to train an initial defect detection model. This ensures that the trained target defect detection model has good defect recognition accuracy. Because the training set is free of interference from defect-free features, the initial defect detection model only needs to learn about defect features during training. This avoids false detections caused by crack-like images due to the surface structure of the part, reducing the false detection rate of the target defect detection model and enabling accurate detection of surface defects on stamped sheet metal parts.

[0014] In one optional implementation, acquiring a first image corresponding to the defect area of ​​a plurality of first stamped sheet metal parts with surface defects and a second image of the corresponding defect-free area on the surface of a standard first stamped sheet metal part includes:

[0015] Obtain a third image of the surface of the first stamped sheet metal part with surface defects and its corresponding defect annotations, and obtain a fourth image of the surface of the standard first stamped sheet metal part.

[0016] Based on the defect annotation, the first image corresponding to the defect region is extracted from the third image, and the defect annotation is determined as the first image for defect annotation;

[0017] The fourth image is image registered with the third image, and the second image is extracted from the image-registered fourth image based on the pixel position of the defect region.

[0018] This invention acquires an overall image of the surface of a defective stamped sheet metal part, and then extracts a first image corresponding to the defect area based on defect annotations. This reduces the amount of image data processing and avoids interference from images of other defect-free areas on the surface of the stamped sheet metal part, providing a more accurate data foundation for subsequent training of the initial defect detection model. This further improves the detection accuracy of the entire target defect detection model. Furthermore, by image registration between the overall image of the defective stamped sheet metal part and the overall image of a standard stamped sheet metal part without defects, a second image corresponding to the defect area is extracted from the overall image of the standard stamped sheet metal part without defects. This ensures the consistency between the second image and the first image, providing an accurate data foundation for subsequent image feature comparison, further improving the sample quality of the training set, and thus further enhancing the detection accuracy of the entire target defect detection model.

[0019] Secondly, the present invention provides a defect detection method, the method comprising:

[0020] A fifth image of the surface of the stamped sheet metal part to be inspected is acquired, and image features are extracted from the fifth image to obtain the fourth image feature data;

[0021] The fifth image feature data is input into the target defect detection model to obtain the defect detection result of the stamped sheet metal part to be detected. The target defect detection model is trained using the defect detection model training method provided by the first aspect and its corresponding optional implementation method. The stamped sheet metal part to be detected is of the same type as the first stamped sheet metal part corresponding to the target defect detection model.

[0022] This invention extracts image features from the surface image of the stamped sheet metal part to be inspected and inputs them into a pre-trained target defect detection model according to another embodiment of the invention. Since the training set used by the target defect detection model during the training process does not contain interference from defect-free features, it avoids false detections of crack-like images caused by the surface structure of the part. This results in a very low false detection rate for the target defect detection model, enabling accurate detection of surface defects on stamped sheet metal parts. This defect detection method can be widely applied in the actual working conditions of surface appearance inspection of stamped sheet metal parts, further improving the inspection and production efficiency of parts.

[0023] In one optional implementation, the step of extracting image features from the fifth image to obtain fourth image feature data includes:

[0024] Compare the fifth image with the sixth image corresponding to the surface of the standard first stamped sheet metal part, and extract the changed areas in the fifth image;

[0025] Image features are extracted from the changed region to obtain fourth image feature data.

[0026] This invention extracts the areas of variation that are inconsistent with the surface image of the stamped sheet metal part to be inspected by comparing it with the surface image of the stamped sheet metal part without defects. These areas of variation are the image regions where defects may exist, thus achieving coarse location of the defects. Then, by extracting image features from the image regions with defects, interference from image features of other image regions without defects is avoided. This provides an accurate data foundation for subsequent defect identification by the target defect detection model, further improving the accuracy of the final defect detection results.

[0027] In one optional implementation, the step of comparing the fifth image with a sixth image corresponding to the surface of a standard first stamped sheet metal part, and extracting the changed areas in the fifth image, includes:

[0028] Divide the fifth image pixel by pixel with the sixth image corresponding to the surface of the standard first stamped sheet metal part to obtain the pixel ratio of each pixel.

[0029] The pixel ratio of each pixel is segmented based on a preset pixel ratio threshold to obtain pixel regions with pixel ratios less than the preset pixel ratio threshold.

[0030] The image region in the fifth image corresponding to the pixel region is determined as the change region.

[0031] This invention calculates the pixel ratio of each pixel in two images by dividing them pixel by pixel, and then obtains the changing region by segmenting the pixel ratio. This allows for the adjustment of the pixel ratio threshold according to the actual working conditions of the stamping sheet metal parts inspection site, thereby achieving accurate extraction of the changing region and further improving the accuracy of the defect region extraction results.

[0032] In an optional implementation, before dividing the fifth image pixel-by-pixel by the sixth image corresponding to the surface of the standard first stamped sheet metal part to obtain the pixel ratio of each pixel, the method further includes:

[0033] Identify the surface contour of the part in the sixth image and the first position of the surface contour of the part in the sixth image, and identify the surface contour of the part in the fifth image and the second position of the surface contour of the part in the fifth image;

[0034] Based on the positional relationship between the first position and the second position, calculate the affine transformation matrix between the fifth image and the sixth image;

[0035] The image position is corrected based on the affine transformation matrix of the fifth image.

[0036] This invention utilizes the similarity of the surface contours of parts to match the fifth and sixth images based on the positional relationship of the surface contours in different images. The affine transformation matrix between the two images is calculated to correct the image position of the fifth image. This ensures the consistency of the actual position of the same pixel when comparing the fifth and sixth images pixel by pixel after position correction, thereby further improving the accuracy of defect area extraction results.

[0037] In an optional implementation, the method further includes:

[0038] Collect template images of the surface of the first stamped sheet metal part under different displacements;

[0039] The fifth image is registered with each template image to obtain the matching rate between the fifth image and each template image;

[0040] The template image with the highest matching rate is selected as the sixth image corresponding to the standard first stamped sheet metal part.

[0041] This invention obtains a template image that is closest to the displacement of the stamped sheet metal surface corresponding to the fifth image by registering it with a template image of the standard stamped sheet metal part surface under different displacements. This is used as the sixth image, thereby avoiding the problem that the difference between the acquired image and the reference image is large due to the displacement of the stamped sheet metal part during the transmission process, which affects the accuracy of subsequent change area extraction. This further improves the accuracy of defect area extraction results.

[0042] Thirdly, the present invention provides a defect detection model training device, the device comprising:

[0043] The first acquisition module is used to acquire a first image corresponding to the defect area of ​​a number of first stamped sheet metal parts with surface defects, and a second image of the defect area corresponding to the defect-free area on the surface of a standard first stamped sheet metal part. The first image is marked with defects.

[0044] The first processing module is used to establish the association between the first image and the second image based on the defect region, forming an image pair;

[0045] The second processing module is used to extract image features from the first image and the second image in each image pair to obtain the first image feature data corresponding to the first image and the second image feature data corresponding to the second image.

[0046] The third processing module is used to remove image features that are the same as those in the second image feature data from the first image feature data to obtain the third image feature data.

[0047] The fourth processing module is used to construct a training set based on the feature data of each of the third images and the defect annotations of the corresponding first images;

[0048] The fifth processing module is used to train the initial defect detection model using the training set to obtain the target defect detection model corresponding to the first stamped sheet metal part.

[0049] Fourthly, the present invention provides a defect detection device, the device comprising:

[0050] The second acquisition module is used to acquire a fifth image of the surface of the stamped sheet metal part to be inspected, and to extract image features from the fifth image to obtain fourth image feature data.

[0051] The sixth processing module is used to input the fifth image feature data into the target defect detection model to obtain the defect detection result of the stamped sheet metal part to be detected. The target defect detection model is trained using the defect detection model training device described in the third aspect. The stamped sheet metal part to be detected is of the same type as the first stamped sheet metal part corresponding to the target defect detection model.

[0052] Fifthly, the present invention provides a defect detection device, comprising: an industrial control computer, wherein the industrial control computer includes:

[0053] The system includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the method of the first aspect or any corresponding embodiment thereof, or to perform the method of the second aspect or any corresponding embodiment thereof.

[0054] In one optional embodiment, the defect detection device further includes an imaging system, which comprises a camera lens assembly and a shadowless light source assembly, wherein...

[0055] The camera lens group is set on the inspection station of the stamped sheet metal parts production line. The camera field of view of the camera lens group covers the surface of the stamped sheet metal parts located at the inspection station, and is used to collect images of the surface of the stamped sheet metal parts and send the collected images to the industrial control computer.

[0056] The shadowless light source group is used to remove the light and shadow in the area where the stamped sheet metal parts are located on the inspection station.

[0057] In a sixth aspect, the present invention provides a computer-readable storage medium storing computer instructions that cause a computer to perform the method of the first aspect or any corresponding embodiment thereof, or to perform the method of the second aspect or any corresponding embodiment thereof.

[0058] Beneficial effects:

[0059] The defect detection model training and defect detection scheme provided by this invention constructs image pairs by combining images of defective areas on the surface of stamped sheet metal parts with corresponding images of defect-free standard stamped sheet metal parts. Simultaneous image feature extraction is performed, and then the image features corresponding to the defective areas are discarded to remove those that overlap with the features of the defect-free areas. This ensures that the retained image features are only related to defects on the surface of the stamped sheet metal parts. These retained image features are used as the training set to train the initial defect detection model. This ensures that the trained target defect detection model has good defect recognition accuracy. Because the training set is free of interference from defect-free features, the initial defect detection model only needs to learn about defect features during training, avoiding false detections due to crack-like images caused by the surface structure of the parts, thus reducing the false detection rate. Furthermore, by extracting image features from the surface image of the stamped sheet metal part to be inspected and inputting it into the trained target defect detection model, accurate detection of surface defects on stamped sheet metal parts can be achieved. This defect detection method can be widely applied in the actual working conditions of surface appearance inspection of stamped sheet metal parts, further improving the inspection and production efficiency of parts. Attached Figure Description

[0060] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0061] Figure 1 This is a schematic diagram of the channel-type structural layout of a defect detection device according to an embodiment of the present invention;

[0062] Figure 2 This is a schematic diagram of the robotic arm structure layout of a defect detection device according to an embodiment of the present invention;

[0063] Figure 3 This is a schematic diagram of the robotic arm assembly structure of the defect detection device according to an embodiment of the present invention;

[0064] Figure 4This is a schematic diagram of the imaging system of a defect detection device according to an embodiment of the present invention;

[0065] Figure 5 This is a flowchart illustrating the defect detection model training method according to an embodiment of the present invention;

[0066] Figure 6 This is another flowchart illustrating the defect detection model training method according to an embodiment of the present invention;

[0067] Figure 7 This is a flowchart illustrating a defect detection method according to an embodiment of the present invention;

[0068] Figure 8 This is another schematic flowchart of a defect detection method according to an embodiment of the present invention;

[0069] Figure 9 This is a schematic diagram illustrating the specific working process of the defect detection method according to an embodiment of the present invention;

[0070] Figure 10 This is a structural block diagram of a defect detection model training device according to an embodiment of the present invention;

[0071] Figure 11 This is a structural block diagram of a defect detection device according to an embodiment of the present invention;

[0072] Figure 12 This is a schematic diagram of the hardware structure of the industrial control computer of the defect detection equipment according to an embodiment of the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0074] In recent years, with the rapid development of machine vision technology, especially in the field of industrial defect detection, the technology of defect identification using deep learning algorithms such as target detection and change detection has become increasingly mature and its detection capability has been significantly improved. In this field, commonly used methods for detecting surface defects in automotive sheet metal include acquiring defect slice data using infrared methods, training a ResNet50 network using the training weights of a VGG network as the initial parameters, and classifying defects by learning the features of defect and non-defect samples. Defect detection and defect localization are implemented in two steps. Another method involves using a teacher model and three student models to learn defect-free samples and outputting abnormal regions, but this approach cannot obtain reliable defect classification. Furthermore, since stamped sheet metal parts often have irregular shapes and sharp edges, the image features of these structures are similar to cracks. Detection models using traditional deep learning algorithms for defect identification often identify these structures as cracks, resulting in a large number of false detections and preventing practical application in the defect detection of stamped sheet metal parts.

[0075] This invention provides a defect detection model training and detection scheme. By comparing the features of defect-free samples with those of defective samples, it greatly reduces false detections of cracks caused by the part structure, while enhancing the detection model's ability to identify crack features, thus improving the model's detection accuracy and detection rate. Furthermore, by employing target detection and template matching-based target detection models, it can quickly achieve one-step defect detection and location. This invention provides a visual online detection scheme for surface appearance defects of stamped sheet metal parts, which can detect and locate sheet metal defects such as scratches, dents, and cracks.

[0076] According to embodiments of the present invention, a defect detection device is provided for detecting whether there are defects in the surface appearance quality of stamped sheet metal parts. In practical applications, this defect detection device can be applied to the inspection station for surface appearance defects of automotive stamped sheet metal parts, to perform surface appearance defect detection on stamped sheet metal parts such as car door inner and outer panel parts. Figure 1 As shown, the defect detection equipment can be configured as a channel type, specifically including: an imaging system and an industrial control computer (…). Figure 1 (Not shown in the image), wherein the industrial control computer is installed in the industrial control cabinet 3. In practical applications, this industrial control computer can be the industrial control computer of the automotive stamping sheet metal parts production line. The specific working principle and working process of the industrial control computer can be found in the relevant description of the method embodiment below, and will not be repeated here. The imaging system includes: a shadowless light source group 1 and a camera lens group 2, wherein the camera lens group 2 is installed at the inspection station of the stamping sheet metal parts production line. The camera field of view of the camera lens group 2 covers the surface of the stamping sheet metal parts located at the inspection station, and is used to collect images of the surface of the stamping sheet metal parts and send the collected images to the industrial control computer; the shadowless light source group is used to remove the light and shadow in the area where the stamping sheet metal parts are located at the inspection station.

[0077] Specifically, in practical applications, such as Figure 2 As shown, the aforementioned defect detection equipment 201 can also be configured as a robotic arm and installed at the inspection station of the automotive stamping sheet metal production line. For example, when the car door outer panel part arrives at the inspection station, the industrial control computer controls the camera lens group 2 and the shadowless light source group 1 to turn on, collect images of the surface of the car outer panel part, and send the collected images to the industrial control computer. The industrial control computer uses the trained defect detection model to perform surface appearance defect detection, thereby detecting whether there are appearance defects in the car outer part, and outputting the detection results to determine whether the car outer panel part is OK or NG.

[0078] Furthermore, to increase the flexibility of the defect detection equipment and ensure that it is unaffected by part slippage or changes in part type, camera lens group 2 provides full field of view coverage of the target area where the target stamped sheet metal part is located. The number of camera lenses (N) in camera lens group 2 is determined by the target area (S), the field of view of a single camera lens (s), the system accuracy (m), and the camera resolution (F). The relationships between these parameters are as follows:

[0079] Number of camera lenses N = n * target area (S) / field of view of a single camera lens (S);

[0080] The field of view of a single camera lens (s) = camera resolution (F) * system accuracy (m);

[0081] The target area is a rectangular area that completely covers the target stamped sheet metal part at the inspection station, including the total area of ​​the five sides: front, back, left, right, and top. Considering factors such as cost and installation difficulty, a larger field of view for each camera is better. The system accuracy (m) is the accuracy of this imaging system, meeting the requirements for appearance defect detection accuracy in automotive sheet metal production lines. The optimal system accuracy and appearance defect detection accuracy satisfy the one-tenth rule.

[0082] In particular, to reduce the number of system hardware components, a single camera lens can be used to perform the functions of multiple camera lenses as described above through a servo mechanism or robotic arm.

[0083] In practical applications, when inspecting different types of stamped sheet metal parts, the industrial control computer calls different camera lenses to capture images. The camera group is called according to the inspection area of ​​different parts. Taking the inner and outer panels of a car door as an example, the outer panel of the car door has a relatively simple shape, and the camera field of view mainly covers the plane of the part, which can be achieved by calling the top camera. For the inner panel of the car door, the field of view needs to cover the plane of the part as well as the vertical surfaces in the left, right and rear directions, which requires calling the top and the cameras in the left, right and rear directions.

[0084] Among them, the shadowless light source group 1 is a rectangular area that completely covers the target part at the inspection station, including five groups of surface light sources: front, back, left, right, and top. Specifically, if the overall layout adopts a robotic arm structure, such as... Figure 3 As shown, a robotic arm 4 is connected to an imaging system 5. The image acquisition range and angle of the camera lens assembly in the imaging system 5 are adjusted by controlling the movement of the robotic arm 4. For example, as shown... Figure 4 As shown, a shadowless light box 502 and a camera lens group 2 can be set on the tooling 501 of the stamped sheet metal parts to realize image acquisition of the surface of the stamped sheet metal parts. Through the shadowless light box 502, the target area can be fully illuminated to form a shadowless light effect.

[0085] The defect detection equipment provided in this invention, whether channel-type or robotic arm-type, can be directly applied to the inspection station on the production line of stamped sheet metal parts. By setting up an imaging system, it automatically acquires images of the surface of the stamped sheet metal parts to be tested. The images acquired by the imaging system are processed by the existing industrial control computer on the production line, thereby realizing the automatic detection of appearance defects of stamped sheet metal parts. This further improves the production efficiency and inspection efficiency of the entire stamped sheet metal parts production line. Moreover, it eliminates the need for manual defect inspection, saving labor costs, and further improves the automation and intelligence of the entire production line.

[0086] According to an embodiment of the present invention, a defect detection model training method embodiment is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0087] This embodiment provides a defect detection model training method, which can be used for, for example Figure 1 The defect detection equipment shown is equipped with an industrial control computer, such as a microcontroller or PLC. Figure 5 This is a flowchart of a defect detection model training method according to an embodiment of the present invention, such as... Figure 5 As shown, the process includes the following steps:

[0088] Step S501: Obtain a first image corresponding to the defect area of ​​a number of first stamped sheet metal parts with surface defects, and a second image of the defect area corresponding to the defect-free area on the surface of a standard first stamped sheet metal part.

[0089] The first image includes defect annotations. These annotations include the location and type of the defect in the image. The location can be indicated by a rectangular frame in the image. The defect types include, but are not limited to, general defects and cracking defects. General defects refer to defects such as dents, scratches, dezincification, and burrs. Cracking defects refer to obvious cracks, hidden cracks, and necking. This invention is not limited to these categories.

[0090] Specifically, the first image is an image area corresponding to the location of appearance defects on the stamped sheet metal part within a certain range, the second image is an image of the same stamped sheet metal part surface area as the first image on a standard stamped sheet metal part image without any appearance defects, and the second image is a standard template corresponding to the first image.

[0091] Step S502: Establish the association between the first image and the second image based on the defect area to form an image pair.

[0092] Specifically, the first image corresponding to the same defect area and the second image of the defect area corresponding to the defect-free area on the surface of the standard stamped sheet metal part are combined into an image pair. The defect image and the defect-free image, which belong to the same area on the surface of the stamped sheet metal part, are paired to facilitate comparison of the image differences caused by appearance defects.

[0093] Step S503: Extract image features from the first image and the second image in each image pair to obtain the first image feature data corresponding to the first image and the second image feature data corresponding to the second image.

[0094] Specifically, the image feature extraction method is an existing technology, and can be implemented using existing image feature extraction methods or feature extraction networks such as backbone networks. This invention is not limited to these. The extracted image features include, but are not limited to, contour features and size features.

[0095] Step S504: Remove image features in the first image feature data that are the same as those in the second image feature data to obtain the third image feature data.

[0096] Specifically, for each image feature in the first image feature data, if the image feature also exists in the second image feature data, it means that the image feature is unrelated to the defect and should be removed from the first image feature data. Conversely, if the image feature is not in the second image feature data, it means that the image feature may be related to the defect and should be retained as a subsequent training sample to construct the training set.

[0097] Step S505: Construct a training set based on the feature data of each third image and the defect annotations of the corresponding first image.

[0098] Specifically, in practical applications, several new images can be obtained by translating, rotating, mapping, adding noise, etc., on the first image. These new images are then used as the first image to perform the above step S501, thereby enriching the training sample size of the training set through this data augmentation.

[0099] Step S506: Train the initial defect detection model using the training set to obtain the target defect detection model corresponding to the first stamped sheet metal part.

[0100] The initial defect detection model can be a machine learning model from existing technologies. For example, in this embodiment, the feature extraction and feature selection processes in steps S503 and S504 are implemented using a portion of the initial defect detection model's structure. The main body of this initial defect detection model consists of a Backbone side for basic feature extraction, a Nick side for high-level and low-level feature fusion, and a Head side with multiple detection heads. The Backbone side consists of convolutional layers, batch normalization layers, and Silu layers, with the 3x3 convolution structure employing the best optimization effect. The Nick side uses upsampling and downsampling to form a feature pyramid from the extracted basic image features, connecting and fusing the rich features of high-level semantic information and the accurate features of low-level target location. The Head side introduces an auxiliary head for deep supervision of model training, ultimately obtaining the defect category and location information as the output. The Backbone side for basic feature extraction is used to implement steps S503 and S504. The training sample set input to the entire initial defect detection model during training is the first and second images mentioned above; this is merely an example and is not intended to limit the invention.

[0101] The defect detection model training method provided in this invention forms an image pair by combining an image of a defective area on the surface of a stamped sheet metal part with an image of the corresponding area of ​​a standard stamped sheet metal part without defects. Simultaneous image feature extraction is performed, and then the image features corresponding to the defective area are discarded to remove any overlap with the image features of the defect-free area. This ensures that the retained image features are only related to defects on the surface of the stamped sheet metal part. The retained image features are then used as a training set to train the initial defect detection model. This ensures that the trained target defect detection model has good defect recognition accuracy. Since there is no interference from defect-free features in the training set, the initial defect detection model only needs to learn about defect features during training. This avoids false detections of crack-like images caused by the surface structure of the part, reducing the false detection rate of the target defect detection model and enabling accurate detection of surface defects on stamped sheet metal parts.

[0102] This embodiment provides a defect detection model training method, which can be used for, for example Figure 1 The defect detection equipment shown is equipped with an industrial control computer, such as a microcontroller or PLC. Figure 6 This is a flowchart of a defect detection model training method according to an embodiment of the present invention, such as... Figure 6 As shown, the process includes the following steps:

[0103] Step S601: Obtain a first image corresponding to the defect area of ​​a number of first stamped sheet metal parts with surface defects, and a second image of the defect area corresponding to the defect-free area on the surface of a standard first stamped sheet metal part. The first image has defect annotations.

[0104] Specifically, step S601 includes:

[0105] Step S6011: Obtain a third image of the surface of the first stamped sheet metal part with surface defects and its corresponding defect annotations, and obtain a fourth image of the surface of the standard first stamped sheet metal part.

[0106] Specifically, the third and fourth images are the overall images of the surface of the stamped sheet metal part with surface defects and the overall images of the surface of the stamped sheet metal part without surface defects, respectively.

[0107] Step S6012: Extract the first image corresponding to the defect region from the third image based on the defect annotation, and determine the defect annotation as the first image for defect annotation.

[0108] Specifically, the first image can be extracted by the position of the annotation box corresponding to the defect annotation in the third image.

[0109] Step S6013: Perform image registration between the fourth image and the third image, and extract the second image from the image-registered fourth image based on the pixel position of the defect area.

[0110] Specifically, the third and fourth images may have discrepancies due to factors such as the camera's acquisition angle. By performing image registration on the two images, the second image corresponding to the defect area in the fourth image can be extracted. The specific method of image registration can be implemented using the existing image registration method that utilizes feature point pairs, or other image registration algorithms can be used. This invention is not limited to these methods.

[0111] This invention acquires an overall image of the surface of a defective stamped sheet metal part, and then extracts a first image corresponding to the defect area based on defect annotations. This reduces the amount of image data processing and avoids interference from images of other defect-free areas on the surface of the stamped sheet metal part, providing a more accurate data foundation for subsequent training of the initial defect detection model. This further improves the detection accuracy of the entire target defect detection model. Furthermore, by registering the overall image of the defective stamped sheet metal part with an overall image of a standard stamped sheet metal part without defects, a second image corresponding to the defect area is extracted from the overall image of the standard stamped sheet metal part without defects. This ensures the consistency between the second image and the first image, providing an accurate data foundation for subsequent image feature comparison, further improving the sample quality of the training set, and thus further enhancing the detection accuracy of the entire target defect detection model.

[0112] Step S602: Establish the association between the first image and the second image based on the defect region to form an image pair. See details below. Figure 5 The relevant descriptions of step S502 shown will not be repeated here.

[0113] Step S603: Extract image features from the first image and the second image in each image pair to obtain the first image feature data corresponding to the first image and the second image feature data corresponding to the second image. See details below. Figure 5 The relevant description of step S503 shown will not be repeated here.

[0114] Step S604: Remove image features from the first image feature data that are identical to those in the second image feature data to obtain the third image feature data. See details below. Figure 5 The relevant description of step S504 shown will not be repeated here.

[0115] Step S605: Construct a training set based on the feature data of each third image and the corresponding defect annotations of the first image. See details below. Figure 5 The relevant descriptions of step S505 shown will not be repeated here.

[0116] Step S606: Train the initial defect detection model using the training set to obtain the target defect detection model corresponding to the first stamped sheet metal part. See details below. Figure 5 The relevant description of step S506 shown will not be repeated here.

[0117] In practical applications, based on the image characteristics of stamped sheet metal parts, these parts often exhibit irregular shapes and sharp edges. The image features of these structures are similar to cracks, leading to numerous false positives. Therefore, the training set for the crack defect model in this method consists of crack data (open cracks, obvious necking, hidden cracks) and defect-free data from the same region. Real crack samples are collected, and a Generative Adversarial Network (GAN) is used to generate data and increase the sample size. Crack features are generated primarily from two dimensions: crack morphology (crack opening, fissure, obvious necking) and crack size (size, length). The generated crack defect data, i.e., the first image mentioned above, is placed in the NG folder, and the crack location is identified through data annotation. The defect-free data, i.e., the second image mentioned above, is placed in the OK folder, and the data is labeled as defect-free. The OK data and NG data are synchronously associated; that is, the crack data image corresponds one-to-one with the OK data, representing different images from the same field of view.

[0118] Because OK data is included in the training set, the model's feature extraction process, including the backbone network and upsampling, simultaneously extracts and identifies both defect-free and cracked features. Since OK and NG data are correlated, the training process can, on the one hand, enhance crack feature recognition by comparing the differences between the two types of features with defect-free features as a reference; on the other hand, it corrects false detections of cracks caused by the part's structure. The model logically correlates OK and NG data. When the input image is cracked, the model identifies and extracts all crack features, including true crack features and falsely detected crack features caused by the part's structure. All extracted features are validated in parallel on both OK and NG data, thus filtering out falsely detected features. Finally, the model outputs the correct crack features and location, accurately determining that the input image is truly cracked. Similarly, when the input image is defect-free, the model extracts all image features and validates them in parallel on both OK and NG data. If no true crack features are found, the image is considered defect-free. This approach significantly reduces false detections of crack-like images caused by the part's surface structure, while simultaneously improving the model's ability to extract and detect crack features, achieving accurate detection of cracked areas.

[0119] This embodiment provides a defect detection method, which can be used for, for example Figure 1 The defect detection equipment shown is equipped with an industrial control computer, such as a microcontroller or PLC. Figure 7 This is a flowchart of a defect detection method according to an embodiment of the present invention, such as... Figure 7 As shown, the process includes the following steps:

[0120] Step S701: Obtain the fifth image of the surface of the stamped sheet metal part to be inspected, and extract image features from the fifth image to obtain the fourth image feature data.

[0121] Specifically, the process of extracting image features from the fifth image is similar to the process of extracting image features in step S503 above. For details, please refer to the relevant description of step S503 above, which will not be repeated here.

[0122] Step S702: Input the fifth image feature data into the target defect detection model to obtain the defect detection result of the stamped sheet metal part to be detected.

[0123] The target defect detection model is trained using the defect detection model training method provided in another embodiment of the present invention. The stamped sheet metal part to be detected is of the same type as the first stamped sheet metal part corresponding to the target defect detection model. Since the defect characteristics of different types of stamped sheet metal parts are different, a corresponding target defect detection model can be trained for each type of stamped sheet metal part according to the defect detection model training method provided in another embodiment of the present invention to further improve the accuracy of the detection results.

[0124] Specifically, the defect detection results of the stamped sheet metal part to be inspected include: defect location and defect type. For example, the defect location is the rectangular box marked in the fifth image, and the defect type is a hidden crack.

[0125] The defect detection method provided in this embodiment of the invention extracts image features from the surface image of the stamped sheet metal part to be detected and inputs them into a target defect detection model trained in another embodiment of the invention. Since the training set used by the target defect detection model during the training process does not contain interference from defect-free features, it avoids false detections of crack-like images caused by the surface structure of the part. This results in a very low false detection rate for the target defect detection model, enabling accurate detection of surface defects on stamped sheet metal parts. This defect detection method can be widely applied in the actual working conditions of surface appearance inspection of stamped sheet metal parts, further improving the inspection and production efficiency of parts.

[0126] This embodiment provides a defect detection method, which can be used for, for example Figure 1 The defect detection equipment shown is equipped with an industrial control computer, such as a microcontroller or PLC. Figure 8 This is a flowchart of a defect detection method according to an embodiment of the present invention, such as... Figure 8 As shown, the process includes the following steps:

[0127] Step S801: Obtain the fifth image of the surface of the stamped sheet metal part to be inspected, and extract image features from the fifth image to obtain the fourth image feature data.

[0128] Specifically, step S801 includes:

[0129] Step S8011: Compare the fifth image with the sixth image corresponding to the surface of the standard first stamped sheet metal part, and extract the changed areas in the fifth image.

[0130] Specifically, step S8011 includes:

[0131] Step a1: Divide the fifth image pixel by pixel with the sixth image corresponding to the surface of the standard first stamped sheet metal part to obtain the pixel ratio of each pixel.

[0132] Step a2: Based on a preset pixel ratio threshold, perform threshold segmentation on the pixel ratio of each pixel to obtain the pixel region where the pixel ratio is less than the preset pixel ratio threshold.

[0133] Specifically, since the pixels in the cracked feature area become smaller (white becomes black), when the pixel ratio is close to 1, the image is judged to have not changed significantly. If the ratio is significantly less than 1, the image is judged to have changed significantly. By traversing the image pixel by pixel to connect the regions with similar ratios, the region with obvious ratio changes can be segmented by using a threshold method.

[0134] For example, the fifth image is used as input source A. i The sixth image is used as the standard template A0, and the threshold for determining the change region is: b = a * A i (x,y,z) / A0(x,y,z), where x, y, and z are pixel coordinates, a is the matching rate between the input source and the standard template, and the preset pixel ratio threshold is an empirical value for on-site debugging. Initial debugging can be based on the positioning determination threshold. Usually, the preset pixel ratio threshold is greater than or equal to 0.6. The threshold adjustment is related to information such as on-site brightness and grayscale of the part surface.

[0135] Step a3: Determine the image region corresponding to the pixel region in the fifth image as the change region.

[0136] This invention calculates the pixel ratio of each pixel in two images by dividing them pixel by pixel, and then obtains the changing region by segmenting the pixel ratio. This allows for the adjustment of the pixel ratio threshold according to the actual working conditions of the stamping sheet metal parts inspection site, thereby achieving accurate extraction of the changing region and further improving the accuracy of the defect region extraction results.

[0137] Furthermore, in some optional embodiments, before performing step a1, step S8011 further includes:

[0138] Step b1: Collect template images of the surface of the standard first stamped sheet metal part under different displacements.

[0139] Specifically, during the debugging process of the aforementioned defect detection equipment, images of qualified parts can be acquired under the same field of view as a reference comparison standard. The vision system acquires data under stable conditions, and the reference data is stable. Since stamped sheet metal parts are transported by belt, the parts will undergo a certain controllable displacement. It is necessary to acquire standard references under different displacement conditions, namely the aforementioned template images. At the same time, different references are placed in the same folder.

[0140] Step b2: Register the fifth image with each template image to obtain the matching rate between the fifth image and each template image.

[0141] Specifically, the image to be inspected, namely the fifth image mentioned above, is registered with multiple standard references using a template matching and positioning method. Once the matching rate reaches 60% or more, subsequent image comparisons can be performed. The image to be inspected and multiple compliant standard references are calculated in parallel and synchronously.

[0142] Step b3: The template image with the highest matching rate is determined as the sixth image corresponding to the standard first stamped sheet metal part.

[0143] Specifically, in order to further improve the accuracy of the defect area extraction results, the template image with the highest matching rate can be determined as the sixth image corresponding to the standard first stamped sheet metal part.

[0144] This invention obtains a template image that is closest to the displacement of the stamped sheet metal surface corresponding to the fifth image by registering it with a template image of the standard stamped sheet metal part surface under different displacements. This is used as the sixth image, thereby avoiding the problem that the difference between the acquired image and the reference image is large due to the displacement of the stamped sheet metal part during the transmission process, which affects the accuracy of subsequent change area extraction. This further improves the accuracy of defect area extraction results.

[0145] Step b4: Identify the surface contour of the part in the sixth image and its first position in the sixth image, and identify the surface contour of the part in the fifth image and its second position in the fifth image.

[0146] Specifically, when production is stable, the part is positioned within the field of view of the acquisition system by adjusting its placement, thus establishing its reference position. A standard matching template, as shown in the sixth image above, is created. Surface features of the part, such as perforations or edges, are selected, and their grayscale contrast is used to obtain the feature contours, which serve as the matching standard template for the part's reference position. Using this standard template as a reference, the matching reference point and reference angle for the standard position can be obtained. When a new part image, as shown in the fifth image above, is input, the standard template contour features are used to traverse the new image for template matching. Template matching is completed through contour similarity, yielding the matching information and position of the new part image.

[0147] Step b5: Based on the positional relationship between the first and second positions, calculate the affine transformation matrix between the fifth and sixth images.

[0148] Step b6: Correct the image position of the fifth image based on the affine transformation matrix.

[0149] Specifically, the affine transformation matrix for position correction is calculated based on the relationship between the two points and two angles of the new position information and the standard position information. The image of the new part obtains the corrected position information through affine transformation, and then the position of the new part image is corrected and registered with the reference to complete the position correction.

[0150] This invention utilizes the similarity of the surface contours of parts to match the fifth and sixth images based on the positional relationship of the surface contours in different images. The affine transformation matrix between the two images is calculated to correct the image position of the fifth image. This ensures the consistency of the actual position of the same pixel when comparing the fifth and sixth images pixel by pixel after position correction, thereby further improving the accuracy of defect area extraction results.

[0151] Step S8012: Extract image features from the changed region to obtain the fourth image feature data.

[0152] Specifically, the process of extracting image features from the changing region is similar to the process of extracting image features in step S503 above. For details, please refer to the relevant description of step S503 above, which will not be repeated here.

[0153] This invention compares the surface image of a stamped sheet metal part to be inspected with an image of a stamped sheet metal part without defects. This process extracts the areas of inconsistency between the two images, identifying potential defective regions. These areas provide a coarse location of the defects. Furthermore, feature extraction of the defective regions avoids interference from features of other defect-free areas, providing a precise data foundation for subsequent defect detection modeling and further improving the accuracy of the final defect detection results.

[0154] Step S802: Input the fifth image feature data into the target defect detection model to obtain the defect detection result of the stamped sheet metal part to be detected. The target defect detection model is trained using the defect detection model training method provided in another embodiment of the present invention. The stamped sheet metal part to be detected is of the same type as the first stamped sheet metal part corresponding to the target defect detection model. For details, please refer to... Figure 7 The relevant description of step S702 shown will not be repeated here.

[0155] It should be noted that when the feature extraction and feature screening processes in steps S503 and S504 are implemented by a portion of the structure of the initial defect detection model, the fifth image can be directly input into the target defect detection model, and the backbone network of the target defect detection model can execute step S701. This invention is not limited thereto.

[0156] The defect detection method provided in this invention will be described in detail below with reference to specific application examples.

[0157] Since existing deep learning-based defect detection models have good recognition effects on general defects such as dents, scratches, dezincification, and burrs, but have a high false detection rate for crack defects such as obvious cracks, hidden cracks, and necking, in this embodiment of the invention, the defect detection process can be divided into two processes: general defect detection and crack defect detection.

[0158] like Figure 9 As shown, the general defect detection process is as follows: First, template matching is used to locate the part. Then, the defect is detected by the target detection model, and the general defect type is classified and the defect location is detected. The target detection model here can be the target defect detection model trained in another embodiment of the present invention, or it can be a detection model in the prior art. The present invention is not limited to this. The detection principle of general defects is as follows: collect general defect data, use the pre-selected detection model to train the model, and obtain the required general defect target detection model. The general defect data mainly consists of general defect samples, including dent and bump data, pressure scratch data, zinc stripping data, burr data, etc., and the sample ratio is approximately 1:1:1:1. Optimally, the total amount of data should be greater than 10,000.

[0159] Furthermore, the crack defect detection process is as follows: First, template matching is used to locate the part. Then, a change detection algorithm is used to obtain the change area that differs from the standard template. The change area is then detected using a target detection model based on template matching to determine the location of the crack defect. This method can greatly eliminate the problems of missed detections and false detections caused by small sample sizes in the target detection algorithm. The target detection principle for crack defects is as follows: Crack defect data is collected, and the defect detection model training scheme provided in this embodiment of the invention is used to train the model, resulting in a crack defect target detection model. The crack defect data mainly consists of crack defect samples and defect-free samples, including obvious crack sample data, hidden crack sample data, necking sample data, and normal sample data, with a sample ratio of approximately 1:1:1:1. Ideally, the total data volume should be greater than 10,000. However, since crack samples are difficult to collect, a total data volume greater than 5,000 can still achieve a certain detection function.

[0160] Furthermore, the ratio of training set to validation set in the two datasets mentioned above is 8:2. The model training process is enriched with data augmentation methods (translation, rotation, contour mapping, noise, etc.). Since the data needs to be preprocessed before training, the images are mainly segmented to meet the image input requirements of the model.

[0161] Specifically, image segmentation aims to increase the pixel ratio of small defects, thereby improving the defect detection rate. Image segmentation preprocessing is performed based on image resolution. Image segmentation is determined by the minimum defect pixel ratio, typically between 2.5% and 5%. For example, if the minimum defect pixel ratio is 60*50, the defect pixel ratio is 2.5%, the original image pixel ratio is 4800*4000, and the segmented image pixel ratio is 2400*2000. This can be segmented in a 2*2 manner. Image segmentation is performed serially, meaning each segmented image is calculated sequentially. The number of segmentations depends on the configuration of the on-site industrial control computer and the production cycle; the number of segmentations should not be too high.

[0162] For example, in a practical application, modifications were made to the line end of a stamping workshop to meet its quality inspection requirements. After assessing the site conditions, such as... Figure 1 and Figure 2 As shown, a channel-style layout is adopted. The on-site requirement is to detect defects with a size ≥1mm. According to visual detection rules, the imaging system of the defect detection equipment is designed with an accuracy of 0.1mm / pixel. Based on the site and accuracy requirements, a global area array visible light camera with a resolution of 25 megapixels, a lens focal length of 12mm, and an aperture of f / 8 is selected. This workshop only produces one car model, totaling 11 types of sheet metal parts. After simulation analysis, it was determined that 86 sets of camera lenses are needed. According to the defect detection equipment provided in this embodiment, a shadowless light box is designed with five large area light sources (front, rear, left, right, and top) to ensure full coverage of the target detection area's field of view and light field.

[0163] The target detection model was labeled using actual part defect samples collected from the field. Among these, crack samples included 295 obvious cracks, 250 hidden cracks, and 320 necking samples, with a sample ratio close to 1:1:1. The final sample size was 6090. General defect samples included 3355 dents / bumps, 3372 scratches, 3824 zinc stripping samples, and 3470 burrs, with a sample ratio close to 1:1:1:1. The final sample size was 21110. The training and validation set ratio was 8:2. The model training process employed data augmentation methods (translation, rotation, contour mapping, noise reduction, etc.) to enrich the model's input. Preprocessing of the data before training was necessary, primarily image segmentation to meet the model's image input requirements.

[0164] The training model was generated after training and then validated on the validation set. The crack detection rate reached 98%, and the general defect detection rate reached 95%, which met the requirements of on-site detection. This proves that the target defect detection model trained by the embodiments of the present invention can be applied.

[0165] For example, in another practical application, to meet the needs of new factory construction, the stamping workshop line tail needs to be automated and unmanned. After discussion and evaluation, if... Figure 3 and Figure 4 As shown, a robotic arm layout is adopted to reduce the number and complexity of hardware components. The on-site requirement is to detect defects ≥1mm in size. Based on visual inspection rules, the imaging system's accuracy is designed to be 0.1mm / pixel. Considering the site and accuracy requirements, a global area array visible light camera with a resolution of 25 megapixels, a lens focal length of 12mm, and an aperture of f / 8 is selected. This workshop only produces three car models, totaling 36 types of sheet metal parts. Using the defect detection equipment provided in this embodiment of the invention, full coverage of the field of view for all parts is achieved. The field of view and imaging points are determined, combined with robot trajectory simulation, and finally, the number of camera lens groups is determined to be 12 groups. To ensure image uniformity, a trapezoidal lightbox is selected to ensure a shadowless effect in the field of view.

[0166] The target detection model was labeled using actual part defect samples collected from the field. Among these, crack samples included 998 obvious cracks, 1202 hidden cracks, and 1100 necking samples, with a sample ratio close to 1:1:1. The final sample size was 12210. General defect samples included 3255 dents / bumps, 4120 scratches, 3812 zinc stripping, and 3120 burrs, with a sample ratio close to 1:1:1:1. The final sample size was 25055. The training and validation set ratio was 8:2. The model training process employed data augmentation methods (translation, rotation, contour mapping, noise reduction, etc.) to enrich the model's input. Preprocessing of the data before training was necessary, primarily image segmentation to meet the model's image input requirements.

[0167] The training model was generated after training and then validated on the validation set. The crack detection rate reached 99%, and the general defect detection rate reached 97%, which met the requirements for on-site detection. This proves that the target defect detection model trained by the embodiments of the present invention can be applied.

[0168] This embodiment also provides a defect detection model training device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the terms "module" and "unit" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0169] This embodiment provides a defect detection model training device, such as... Figure 10 As shown, the defect detection model training device includes:

[0170] The first acquisition module 1001 is used to acquire a first image corresponding to the defect area of ​​a number of first stamped sheet metal parts with surface defects, and a second image of the defect area corresponding to the defect-free area on the surface of a standard first stamped sheet metal part. The first image is marked with defect labels.

[0171] The first processing module 1002 is used to establish a correlation between the first image and the second image based on the defect area, forming an image pair;

[0172] The second processing module 1003 is used to extract image features from the first image and the second image in each image pair to obtain first image feature data corresponding to the first image and second image feature data corresponding to the second image.

[0173] The third processing module 1004 is used to remove image features that are the same as those in the second image feature data from the first image feature data to obtain the third image feature data.

[0174] The fourth processing module 1005 is used to construct a training set based on the feature data of each third image and the defect annotations of the corresponding first image;

[0175] The fifth processing module 1006 is used to train the initial defect detection model using the training set to obtain the target defect detection model corresponding to the first stamped sheet metal part.

[0176] In some optional implementations, the first acquisition module 1001 includes:

[0177] The first processing unit is used to acquire a third image of the surface of the first stamped sheet metal part with surface defects and its corresponding defect annotations, and to acquire a fourth image of the surface of the standard first stamped sheet metal part.

[0178] The second processing unit is used to extract the first image corresponding to the defect region from the third image based on the defect annotation, and to determine the defect annotation as the first image for defect annotation.

[0179] The third processing unit is used to perform image registration between the fourth image and the third image, and to extract the second image from the image-registered fourth image based on the pixel position of the defect region.

[0180] In this embodiment, the defect detection model training device is presented in the form of a functional unit. Here, a unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0181] The further functional descriptions of the above modules and units are the same as those in the corresponding method embodiments described above, and will not be repeated here.

[0182] This embodiment also provides a defect detection device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the terms "module" and "unit" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0183] This embodiment provides a defect detection device, such as... Figure 11 As shown, the defect detection device includes:

[0184] The second acquisition module 1101 is used to acquire a fifth image of the surface of the stamped sheet metal part to be inspected, and to extract image features from the fifth image to obtain fourth image feature data.

[0185] The sixth processing module 1102 is used to input the fifth image feature data into the target defect detection model to obtain the defect detection result of the stamped sheet metal part to be detected. The target defect detection model is trained by the defect detection model training device as described in another embodiment of the present invention. The stamped sheet metal part to be detected is of the same type as the first stamped sheet metal part corresponding to the target defect detection model.

[0186] In some optional implementations, the second acquisition module 1101 includes:

[0187] The fourth processing unit is used to compare the fifth image with the sixth image corresponding to the surface of the standard first stamped sheet metal part, and extract the change area in the fifth image.

[0188] The fifth processing unit is used to extract image features from the changed areas to obtain the fourth image feature data.

[0189] In some optional implementations, the fourth processing unit includes:

[0190] The first processing subunit is used to divide the fifth image and the sixth image corresponding to the surface of the standard first stamped sheet metal part pixel by pixel to obtain the pixel ratio value of each pixel.

[0191] The second processing subunit is used to perform threshold segmentation on the pixel ratio of each pixel based on a preset pixel ratio threshold to obtain pixel regions with pixel ratios less than the preset pixel ratio threshold.

[0192] The third processing subunit is used to determine the image region corresponding to the pixel region in the fifth image as the change region.

[0193] In some optional embodiments, the fourth processing unit further includes:

[0194] The fourth processing subunit is used to identify the surface contour of the part in the sixth image and the first position of the surface contour of the part in the sixth image, and to identify the surface contour of the part in the fifth image and the second position of the surface contour of the part in the fifth image.

[0195] The fifth processing subunit is used to calculate the affine transformation matrix between the fifth image and the sixth image based on the positional relationship between the first position and the second position.

[0196] The sixth processing subunit is used to correct the image position of the fifth image based on the affine transformation matrix.

[0197] In an optional implementation, the second acquisition module 1101 further includes:

[0198] The sixth processing unit is used to acquire template images of the surface of the standard first stamped sheet metal part under different displacements;

[0199] The seventh processing unit is used to register the fifth image with each template image to obtain the matching rate between the fifth image and each template image.

[0200] The eighth processing unit is used to determine the template image with the highest matching rate as the sixth image corresponding to the standard first stamped sheet metal part.

[0201] In this embodiment, the defect detection model training device is presented in the form of a functional unit. Here, a unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0202] The further functional descriptions of the above modules and units are the same as those in the corresponding method embodiments described above, and will not be repeated here.

[0203] Please see Figure 12 , Figure 12 This is a schematic diagram of the structure of an industrial control computer in a defect detection device provided by an optional embodiment of the present invention, as shown below. Figure 12As shown, the industrial control computer includes one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the electronic devices, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple electronic devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 12 Take a processor 10 as an example.

[0204] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0205] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0206] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device as displayed on a mini-program landing page. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0207] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0208] The industrial computer also includes a communication interface 30 for the electronic device to communicate with other devices or communication networks.

[0209] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0210] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A defect detection model training method, characterized in that, The method includes: Acquire a first image corresponding to the defect area of ​​a number of first stamped sheet metal parts with surface defects, and a second image of the defect area corresponding to the defect-free area on the surface of a standard first stamped sheet metal part. The first image is marked with defect labels. Based on the defect region, establish the association between the first image and the second image to form an image pair; Image features are extracted from the first image and the second image in each image pair to obtain the first image feature data corresponding to the first image and the second image feature data corresponding to the second image. Remove image features from the first image feature data that are identical to those in the second image feature data to obtain the third image feature data; A training set is constructed based on the feature data of each third image and the defect annotations of the corresponding first image; The initial defect detection model is trained using the training set to obtain the target defect detection model corresponding to the first stamped sheet metal part.

2. The method according to claim 1, characterized in that, The acquisition of a first image corresponding to the defect area of ​​a plurality of first stamped sheet metal parts with surface defects and a second image of the corresponding defect-free area on the surface of a standard first stamped sheet metal part includes: Obtain a third image of the surface of the first stamped sheet metal part with surface defects and its corresponding defect annotations, and obtain a fourth image of the surface of the standard first stamped sheet metal part. Based on the defect annotation, the first image corresponding to the defect region is extracted from the third image, and the defect annotation is determined as the first image for defect annotation; The fourth image is image registered with the third image, and the second image is extracted from the image-registered fourth image based on the pixel position of the defect region.

3. A defect detection method, characterized in that, The method includes: A fifth image of the surface of the stamped sheet metal part to be inspected is acquired, and image features are extracted from the fifth image to obtain the fourth image feature data; The fifth image feature data is input into the target defect detection model to obtain the defect detection result of the stamped sheet metal part to be detected. The target defect detection model is trained using the defect detection model training method as described in claim 1 or 2. The stamped sheet metal part to be detected is of the same type as the first stamped sheet metal part corresponding to the target defect detection model.

4. The method according to claim 3, characterized in that, The step of extracting image features from the fifth image to obtain fourth image feature data includes: Compare the fifth image with the sixth image corresponding to the surface of the standard first stamped sheet metal part, and extract the changed areas in the fifth image; Image features are extracted from the changed region to obtain fourth image feature data.

5. The method according to claim 4, characterized in that, The step of comparing the fifth image with a sixth image corresponding to the surface of the standard first stamped sheet metal part, and extracting the changed areas in the fifth image, includes: Divide the fifth image pixel by pixel with the sixth image corresponding to the surface of the standard first stamped sheet metal part to obtain the pixel ratio of each pixel. The pixel ratio of each pixel is segmented based on a preset pixel ratio threshold to obtain pixel regions with pixel ratios less than the preset pixel ratio threshold. The image region in the fifth image corresponding to the pixel region is determined as the change region.

6. The method according to claim 5, characterized in that, Before dividing the fifth image pixel-by-pixel by the sixth image corresponding to the surface of the standard first stamped sheet metal part to obtain the pixel ratio of each pixel, the method further includes: Identify the surface contour of the part in the sixth image and the first position of the surface contour of the part in the sixth image, and identify the surface contour of the part in the fifth image and the second position of the surface contour of the part in the fifth image; Based on the positional relationship between the first position and the second position, calculate the affine transformation matrix between the fifth image and the sixth image; The image position is corrected based on the affine transformation matrix of the fifth image.

7. The method according to claim 4, characterized in that, The method further includes: Collect template images of the surface of the first stamped sheet metal part under different displacements; The fifth image is registered with each template image to obtain the matching rate between the fifth image and each template image; The template image with the highest matching rate is selected as the sixth image corresponding to the standard first stamped sheet metal part.

8. A defect detection model training device, characterized in that, The device includes: The first acquisition module is used to acquire a first image corresponding to the defect area of ​​a number of first stamped sheet metal parts with surface defects, and a second image of the defect area corresponding to the defect-free area on the surface of a standard first stamped sheet metal part. The first image is marked with defect labels. The first processing module is used to establish the association between the first image and the second image based on the defect region, forming an image pair; The second processing module is used to extract image features from the first image and the second image in each image pair to obtain the first image feature data corresponding to the first image and the second image feature data corresponding to the second image. The third processing module is used to remove image features that are the same as those in the second image feature data from the first image feature data to obtain the third image feature data. The fourth processing module is used to construct a training set based on the feature data of each of the third images and the defect annotations of the corresponding first images; The fifth processing module is used to train the initial defect detection model using the training set to obtain the target defect detection model corresponding to the first stamped sheet metal part.

9. A defect detection device, characterized in that, The device includes: The second acquisition module is used to acquire a fifth image of the surface of the stamped sheet metal part to be inspected, and to extract image features from the fifth image to obtain fourth image feature data. The sixth processing module is used to input the fifth image feature data into the target defect detection model to obtain the defect detection result of the stamped sheet metal part to be detected. The target defect detection model is trained using the defect detection model training device as described in claim 8. The stamped sheet metal part to be detected is of the same type as the first stamped sheet metal part corresponding to the target defect detection model.

10. A defect detection device, characterized in that, include: Industrial control computer, the industrial control computer includes: A memory and a processor are communicatively connected, the memory storing computer instructions, and the processor executing the computer instructions to perform the method of any one of claims 1 to 2, or the method of any one of claims 3 to 7.

11. The defect detection equipment according to claim 10, characterized in that, Also includes: The imaging system includes: a camera lens assembly and a shadowless light source assembly, wherein, The camera lens group is set on the inspection station of the stamped sheet metal parts production line. The camera field of view of the camera lens group covers the surface of the stamped sheet metal parts located at the inspection station, and is used to collect images of the surface of the stamped sheet metal parts and send the collected images to the industrial control computer. The shadowless light source group is used to remove the light and shadow in the area where the stamped sheet metal parts are located on the inspection station.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of any one of claims 1 to 2, or the method of any one of claims 3 to 7.

Images