Riveting defect identification method, device and equipment

Abstract

The invention discloses a riveting defect identification method, device and equipment. The method comprises the following steps: acquiring a shock absorption tower riveting image marked as no defect in a riveting area; using the unsupervised classification model as a basic network, and training the unsupervised classification model by using the damping tower riveting image to obtain a trained riveting defect identification model; under the condition that an image to be detected is acquired, inputting the image to be detected into a riveting defect identification model, so that the riveting defect identification model performs defect identification on a riveting area of the image to be detected; and under the condition that the riveting defect identification model outputs an abnormal identification result, determining that riveting defects exist in the riveting area in the to-be-detected image. According to the embodiment of the invention, the riveting defect in the to-be-detected image can be effectively identified, and then riveting quality detection is completed.

Description

Technical Field

[0001] The present application belongs to the field of image detection technology, and in particular relates to a riveting defect recognition method, device and equipment. Background Art

[0002] At present, after the shock absorber towers on both sides of the front end of the vehicle are connected to the vehicle body through riveting, due to the riveting process, the nuts at the riveted points of the shock absorber towers and the edges of the riveted plates may be at risk of cracking. Severe cracks may cause the riveted points to break due to stress in subsequent processes or even during use of the vehicle after leaving the factory.

[0003] Therefore, a method for effectively identifying riveting defects is urgently needed in the relevant technology so as to perform riveting quality inspection on all riveting points to eliminate the above-mentioned safety hazards. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a riveting defect recognition method and device thereof, which can effectively recognize the riveting defects in the image to be detected, and then complete the riveting quality detection.

[0005] In a first aspect, an embodiment of the present application provides a riveting defect recognition method, the method comprising: obtaining a shock tower riveted image marked as defect-free in a riveted area; using an unsupervised classification model as a basic network, and training the unsupervised classification model using the shock tower riveted image to obtain a trained riveted defect recognition model; when an image to be detected is obtained, inputting the image to be detected into the riveted defect recognition model so that the riveted defect recognition model performs defect recognition on the riveted area of ​​the image to be detected; when the riveted defect recognition model outputs an abnormal recognition result, determining that a riveted defect exists in the riveted area in the image to be detected.

[0006] In some implementations of the first aspect, the riveted area includes a riveted point nut area and a riveted plate edge area. After obtaining a shock tower riveted image marked as defect-free in the riveted area, the method further includes: dividing the shock tower riveted image into a first image in which the riveted point nut area is defect-free, and a second image in which the riveted plate edge area is defect-free; and training an unsupervised classification model using the shock tower riveted image, including: training the unsupervised classification model using the first image and the second image.

[0007] In some implementations of the first aspect, the unsupervised classification model includes a first classification model and a second classification model, the riveting defect recognition model includes a first classifier and a second classifier, and the unsupervised classification model is trained using the shock tower riveting image, including: using the first image as a training sample of the first classification model to train the first classification model to obtain a first classifier, wherein the first classifier is used to identify nut cracks in a nut area of ​​a riveted point in the image to be detected; using the second image as a training sample of the second classification model to train the second classification model to obtain a second classifier, wherein the second classifier is used to identify plate edge defects in a riveted plate edge area in the image to be detected.

[0008] In some implementations of the first aspect, the abnormal recognition result includes a first abnormal result and / or a second abnormal result. When the riveting defect recognition model outputs an abnormal recognition result, it is determined that a riveting defect exists in a riveted area in the image to be detected, including: when the first abnormal result is output, it is determined that a nut crack exists in a nut area of ​​a riveted point in the image to be detected; when the second abnormal result is output, it is determined that a plate edge defect exists in a riveted plate edge area in the image to be detected.

[0009] In some implementable embodiments of the first aspect, the image to be detected includes multiple riveted point nut areas, the first abnormal result is used to identify the position of the nut crack in the riveted point nut area, and after determining that there is a plate edge defect in the riveted plate edge area in the image to be detected, the method also includes: determining that the plate edge defect is not a plate edge crack when the first abnormal result is not output; when the first abnormal result is output, screening a target nut area that is closest to the riveted plate edge area from multiple riveted point nut areas; and judging whether the plate edge defect is a plate edge crack based on the positional relationship between the nut crack position and the target nut area.

[0010] In some implementable methods of the first aspect, based on the positional relationship between the nut crack position and the target nut area, determining whether the plate edge defect is a plate edge crack includes: when the nut crack position is outside the target nut area, determining that the plate edge defect is not a plate edge crack; when the nut crack position is within the target nut area, adjusting the pixel values ​​of all pixel points in the riveted plate edge area in the image to be detected based on a preset pixel interval corresponding to the plate edge crack; performing contour detection on the defect position in the riveted plate edge area in the adjusted image to be detected to obtain a contour area; and determining that the plate edge defect is a plate edge crack when the contour area meets a preset crack morphology condition.

[0011] In some implementations of the first aspect, based on a preset pixel interval corresponding to a plate edge crack, pixel values ​​of all pixels in a riveted plate edge region in an image to be detected are adjusted, including: obtaining a pixel histogram of the image to be detected; determining pixel values ​​of all pixels in the riveted plate edge region based on the pixel histogram; determining a pixel whose pixel value is within a preset pixel interval as a first pixel, and a pixel other than the first pixel as a second pixel; adjusting the pixel value of the first pixel to a first preset pixel value, and adjusting the pixel value of the second pixel to a second preset pixel value.

[0012] In some implementations of the first aspect, the image to be detected includes multiple riveted plate edge areas, and the preset crack morphology conditions include at least one of the following: a ratio of the circumference of the contour area within the riveted plate edge area to the target length is greater than a first preset ratio threshold; a ratio of the area of ​​the contour area within the riveted plate edge area to the target area is greater than a second preset ratio threshold; wherein the target length is the length of the longest side of the riveted plate edge area where the contour area is located, and the target area is the area of ​​the riveted plate edge area where the contour area is located.

[0013] In a second aspect, an embodiment of the present application provides a riveting defect identification device, which includes: an acquisition module, used to acquire a shock tower riveted image marked as defect-free in a riveted area; a model training module, used to use an unsupervised classification model as a basic network, and train the unsupervised classification model using the shock tower riveted image to obtain a trained riveted defect identification model; an input module, used to input the image to be detected into the riveted defect identification model when the image to be detected is acquired, so that the riveted defect identification model performs defect identification on the riveted area of ​​the image to be detected; a defect identification module, used to determine that there is a riveted defect in the riveted area in the image to be detected when the riveted defect identification model outputs an abnormal identification result.

[0014] In some implementations of the second aspect, the riveted area includes a riveted point nut area and a riveted plate edge area, and the device also includes: a division module, which is used to divide the shock tower riveted image into a first image without defects in the riveted point nut area and a second image without defects in the riveted plate edge area after acquiring the shock tower riveted image marked as defect-free in the riveted area; the model training module includes: a training unit, which is used to train an unsupervised classification model using the first image and the second image.

[0015] In some implementations of the second aspect, the unsupervised classification model includes a first classification model and a second classification model, the riveting defect recognition model includes a first classifier and a second classifier, and the training unit is specifically used to: use the first image as a training sample of the first classification model to train the first classification model to obtain a first classifier, wherein the first classifier is used to identify nut cracks in the nut area of ​​the riveted point in the image to be detected; use the second image as a training sample of the second classification model to train the second classification model to obtain a second classifier, wherein the second classifier is used to identify plate edge defects in the riveted plate edge area in the image to be detected.

[0016] In some implementations of the second aspect, the abnormal recognition result includes a first abnormal result and / or a second abnormal result, and the defect recognition module is specifically used to: when outputting the first abnormal result, determine that a nut crack exists in a nut area of ​​a riveted point in the image to be detected; when outputting the second abnormal result, determine that a plate edge defect exists in a riveted plate edge area in the image to be detected.

[0017] In some implementable embodiments of the second aspect, the image to be detected includes multiple riveted point nut areas, the first abnormal result is used to identify the position of the nut crack in the riveted point nut area, and the device also includes: a defect recognition module, which is also used to determine that the plate edge defect is not a plate edge crack after determining that there is a plate edge defect in the riveted plate edge area in the image to be detected, without outputting the first abnormal result; a screening module, which is used to screen a target nut area that is closest to the riveted plate edge area from multiple riveted point nut areas when the first abnormal result is output; the defect recognition module is also used to determine whether the plate edge defect is a plate edge crack based on the positional relationship between the nut crack position and the target nut area.

[0018] In some possible implementations of the second aspect, the defect recognition module includes: a determination unit, used to determine that the plate edge defect is not a plate edge crack when the nut crack position is outside the target nut area; an adjustment unit, used to adjust the pixel values ​​of all pixel points in the riveted plate edge area in the image to be detected based on a preset pixel interval corresponding to the plate edge crack when the nut crack position is within the target nut area; a detection unit, used to perform contour detection on the defect position in the riveted plate edge area in the adjusted image to be detected to obtain a contour area; the determination unit is also used to determine that the plate edge defect is a plate edge crack when the contour area meets a preset crack morphology condition.

[0019] In some implementation methods of the second aspect, the adjustment unit is specifically used to: obtain a pixel histogram of the image to be detected; determine the pixel values ​​of all pixels in the riveted plate edge area based on the pixel histogram; determine the pixel points whose pixel values ​​are in a preset pixel interval as first pixels, and the pixel points other than the first pixels as second pixels; adjust the pixel value of the first pixel to a first preset pixel value, and adjust the pixel value of the second pixel to a second preset pixel value.

[0020] In some implementations of the second aspect, the image to be detected includes multiple riveted plate edge areas, and the preset crack morphology conditions include at least one of the following: a ratio of the circumference of the contour area within the riveted plate edge area to the target length is greater than a first preset ratio threshold; a ratio of the area of ​​the contour area within the riveted plate edge area to the target area is greater than a second preset ratio threshold; wherein the target length is the length of the longest side of the riveted plate edge area where the contour area is located, and the target area is the area of ​​the riveted plate edge area where the contour area is located.

[0021] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the steps of the riveting defect identification method of the first aspect are implemented.

[0022] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the steps of the riveting defect identification method of the first aspect are implemented.

[0023] In a fifth aspect, an embodiment of the present application provides a computer program product, which is stored in a non-volatile storage medium and is executed by at least one processor to implement the steps of the riveting defect identification method of the first aspect.

[0024] In a sixth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run a program or instruction to implement the steps of the riveting defect identification method of the first aspect.

[0025] The present application provides a riveting defect recognition method, device and equipment, which obtains a shock tower riveted image marked as defect-free in the riveted area, and on this basis, uses an unsupervised classification model as a basic network, and uses the shock tower riveted image to train the unsupervised classification model, so as to obtain a trained riveting defect recognition model. In this way, when the image to be detected is obtained, the image to be detected is directly input into the riveting defect recognition model, and the riveting defect recognition model can perform defect recognition on the riveted area of ​​the image to be detected. When the riveting defect recognition model outputs an abnormal recognition result, it is determined that there is a riveting defect in the riveted area of ​​the image to be detected. Therefore, the riveting defect recognition method only needs to be trained with a defect-free shock tower riveted image, and the training task can be completed without defective samples. The performance is better, and the defects in the riveted area in the image to be detected can be effectively identified, and the riveting quality inspection can be completed. It can be applied to actual industrial scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application.

[0027] Figure 1 It is a flowchart of a riveting defect identification method provided by an embodiment of the present application;

[0028] Figure 2 is an exemplary schematic diagram of a shock tower riveting image provided in an embodiment of the present application;

[0029] Figure 3 is a flow chart of a riveting defect identification method provided by another embodiment of the present application;

[0030] Figure 4 is a flow chart of a riveting defect identification method provided in yet another embodiment of the present application;

[0031] Figure 5 is an exemplary schematic diagram of a non-defective image, a defective image, and a cracked image provided by an embodiment of the present application;

[0032] Figure 6 is a flow chart of a riveting defect identification method provided in yet another embodiment of the present application;

[0033] Figure 7 is a flow chart of a riveting defect identification method provided in yet another embodiment of the present application;

[0034] Figure 8 is a schematic diagram of an example of a riveting defect identification process provided in an embodiment of the present application;

[0035] Fig. 9 is an exemplary schematic diagram of the image equalization processing effect provided by an embodiment of the present application;

[0036] Fig.10 It is a structural schematic diagram of a riveting defect identification device provided in an embodiment of the present application;

[0037] Fig.11 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0039] Unsupervised classification / detection: learning only through normal samples, outputting the category of the image or the location of the target;

[0040] Riveting cracks: Cracks in the nuts at the riveting points of the shock tower and cracks on the edge of the plate caused by the pressure riveting process.

[0041] In the related art, after the shock absorber towers on both sides of the front end of the vehicle are connected to the vehicle body by riveting, due to the riveting process, the nuts at the riveted points of the shock absorber towers and the edges of the riveted plates may be at risk of cracking. Severe cracks may cause the riveted points to break due to stress in subsequent processes or even during use of the vehicle after leaving the factory. Therefore, it is necessary to detect whether all riveted areas have cracks. A method for effectively identifying riveting defects is urgently needed in the related art so that riveting quality inspection can be performed on all riveted points to eliminate the above-mentioned safety hazards.

[0042] The following describes in detail the riveting defect identification method provided by the embodiment of the present application through specific embodiments and application scenarios in conjunction with the accompanying drawings.

[0043] Figure 1 It is a flowchart of a riveting defect identification method provided in an embodiment of the present application. The executor of the riveting defect identification method may be an electronic device.

[0044] The following takes the rivet defect identification method as an example in which the execution subject is an electronic device to illustrate the rivet defect identification method of the present application. It should be noted that the above execution subject and application scenario do not constitute a limitation on the present application.

[0045] like Figure 1 As shown, the riveting defect identification method provided in the embodiment of the present application may include steps 110 to 140.

[0046] Step 110, obtaining a riveted image of the shock tower marked as defect-free in the riveted area;

[0047] Step 120, using the unsupervised classification model as a basic network, and using the shock tower riveting image to train the unsupervised classification model to obtain a trained riveting defect recognition model;

[0048] Step 130, when the image to be detected is acquired, the image to be detected is input into the riveting defect recognition model, so that the riveting defect recognition model performs defect recognition on the riveting area of ​​the image to be detected;

[0049] Step 140: When the riveting defect recognition model outputs an abnormal recognition result, it is determined that a riveting defect exists in the riveting area of ​​the image to be detected.

[0050] The riveting defect recognition method provided in the embodiment of the present application obtains a shock tower riveted image marked as defect-free in the riveted area. On this basis, an unsupervised classification model is used as a basic network, and the shock tower riveted image is used to train the unsupervised classification model to obtain a trained riveting defect recognition model. In this way, when the image to be detected is obtained, the image to be detected is directly input into the riveting defect recognition model, and the riveting defect recognition model can perform defect recognition on the riveted area of ​​the image to be detected. When the riveting defect recognition model outputs an abnormal recognition result, it is determined that there is a riveting defect in the riveted area of ​​the image to be detected. Therefore, the riveting defect recognition method only needs to be trained with a defect-free shock tower riveted image, and the training task can be completed without defective samples. The performance is better, and the defects in the riveted area in the image to be detected can be effectively identified to complete the riveting quality inspection, which can be applied to actual industrial scenarios.

[0051] The specific implementation of the above steps will be described in detail below in conjunction with specific embodiments.

[0052] Referring to step 110 , an image of the shock tower rivets marked as defect-free in the rivet area is obtained.

[0053] Specifically, the riveted area can be a region of interest (ROI). A shock tower riveted image can include multiple riveted areas. The riveted area can be divided into a nut area of ​​the shock tower riveted point and a riveted plate edge area. The riveted area marked as defect-free means that there are no cracks or defects in the riveted area.

[0054] For example, Figure 2 As shown, there are 8 nut areas and multiple riveted plate edge areas located at the edge of the plate in the riveted image of the shock tower, and there are no defects in these areas.

[0055] In some embodiments, the number of shock tower riveting images to be acquired can be set according to demand, for example, set to 1000 to 5000.

[0056] In step 120, the unsupervised classification model is used as a basic network, and the unsupervised classification model is trained using the riveting image of the shock tower to obtain a trained riveting defect recognition model.

[0057] Specifically, the unsupervised classification model is a pure classification model. By inputting a large number of defect-free shock tower riveted images into the unsupervised model, the unsupervised model can learn certain feature representations from a large amount of data based on deep learning visual methods during the training process, and then perform recognition and prediction in similar samples. The structure and weight parameters of the trained unsupervised classification model are saved to obtain a model that can classify whether an image has riveting defects, that is, a riveting defect recognition model, which solves the difficulty of defect-free samples in industrial scenarios.

[0058] In some embodiments of the present application, in order to effectively identify defects in both the riveted point nut area and the riveted plate edge area, the riveted area may include the riveted point nut area and the riveted plate edge area. Figure 3 is a flow chart of a riveting defect identification method provided by another embodiment of the present application, which may include after step 110 Figure 3 As shown in step 310 , step 130 may include step 320 .

[0059] Step 310, dividing the shock tower riveting image into a first image in which the riveted point nut area has no defects, and a second image in which the riveted plate edge area has no defects;

[0060] Step 320: train an unsupervised classification model using the first image and the second image.

[0061] Specifically, by using the first image to train the unsupervised classification model, the unsupervised classification model can learn the feature representation of the defect-free rivet point nut area from a large number of first images, thereby enabling the unsupervised classification model to have the ability to identify defects from the rivet point nut area; by using the second image to train the unsupervised classification model, the unsupervised classification model can learn the feature representation of the defect-free rivet plate edge area from a large number of second images, thereby enabling the unsupervised classification model to have the ability to identify defects from the rivet plate edge area.

[0062] In some embodiments of the present application, the first image and the second image may be used to train the same unsupervised classification model to obtain a riveting defect recognition model that can simultaneously recognize nut cracks and plate edge defects.

[0063] In some other embodiments of the present application, different unsupervised classification models may be trained using the first image and the second image, the unsupervised classification model includes a first classification model and a second classification model, the riveting defect recognition model includes a first classifier and a second classifier, and the above step 320 may include the following steps:

[0064] Using the first image as a training sample of the first classification model to train the first classification model to obtain a first classifier;

[0065] The second image is used as a training sample of the second classification model to train the second classification model to obtain a second classifier.

[0066] The first classifier is used to identify nut cracks in the nut area of ​​the riveted point in the image to be detected, and the second classifier is used to identify plate edge defects in the riveted plate edge area in the image to be detected.

[0067] In this way, by dividing the shock tower riveting image into a first image with no defects in the riveted point nut area and a second image with no defects in the riveted plate edge area, the first image and the second image can be used to train different unsupervised classification models respectively, and a first classifier with the ability to identify nut cracks and a second classifier with the ability to identify plate edge defects can be obtained, thereby improving the adaptability of different classifiers to defect categories. One classifier is used to classify defects in riveted point nuts, and the other is used to classify defects in the riveted plate edge area, thereby improving the recognition rate. In addition, by providing this detection method, it is possible to build an automatic detection system, and then the automation level and efficiency of the inspection stations involved can be improved by deploying the system on the production line, thereby achieving the purpose of increasing efficiency.

[0068] In step 130, when the image to be detected is acquired, the image to be detected is input into the riveting defect recognition model, so that the riveting defect recognition model performs defect recognition on the riveting area of ​​the image to be detected.

[0069] Specifically, when the riveting defect recognition model is obtained by training the same supervised classification model using the first image and the second image, the electronic device can directly input the image to be detected into the riveting defect recognition model; when the riveting defect recognition model includes a first classifier and a second classifier, the electronic device can input the image to be detected into the first classifier and the second classifier respectively.

[0070] In some embodiments, the image to be inspected and the shock tower riveted image may be taken by the same camera, so that the camera may be used to calibrate multiple ROI positions in the image, and all riveted point nut areas and riveted plate edge areas in the image to be inspected and the shock tower riveted image may be divided based on the multiple ROI positions.

[0071] In step 140 , when the riveting defect recognition model outputs an abnormal recognition result, it is determined that a riveting defect exists in the riveting area of ​​the image to be detected.

[0072] In some embodiments of the present application, the abnormality identification result may include a first abnormality result and / or a second abnormality result. Figure 4 is a flow chart of a riveting defect identification method provided by another embodiment of the present application. Step 140 may include Figure 4 Steps 410 and 420 are shown.

[0073] Step 410, when the first abnormal result is output, determining that a nut crack exists in the nut area of ​​the riveted point in the image to be detected;

[0074] Step 420: When the second abnormal result is output, it is determined that there is a plate edge defect in the riveted plate edge region in the image to be detected.

[0075] It should be noted that step 410 and step 420 correspond to two situations, and the order of execution may not be distinguished; when the first abnormal result and the second abnormal result are output, it is determined that there is a nut crack in the nut area of ​​the riveted point in the image to be detected, and there is a plate edge defect in the riveted plate edge area.

[0076] Since the actual cracks on the riveted points are relatively large and are not prone to interference defects such as scratches, vinyl, and handwriting, the unsupervised classification model for the nuts at the riveted points can directly classify the real nut cracks; and since the riveted plate edge area is prone to interference defects such as scratches, vinyl, and handwriting, the riveted defect recognition model can only classify the plate edge defects for the plate edge area, but cannot classify the real plate edge cracks.

[0077] For example, Figure 5 As shown in the figure, there are example images of a non-defective image, a defective image and a cracked image. For the defective image, it is necessary to further judge it to determine whether it is a real crack.

[0078] In some embodiments of the present application, in order to classify the plate edge defects and identify the real plate edge cracks, the image to be detected may include multiple riveting point nut areas, and the first abnormal result is used to identify the nut crack position in the riveting point nut area. Figure 6 FIG. 1 is a flow chart of a riveting defect identification method provided by another embodiment of the present application. Figure 6 As shown, after step 420 , steps 610 to 630 may be included.

[0079] Step 610, when the first abnormal result is not output, determining that the plate edge defect is not a plate edge crack;

[0080] Step 620, when the first abnormal result is output, select a target nut area that is closest to the riveted plate edge area from the multiple riveted point nut areas;

[0081] Step 630 , based on the positional relationship between the nut crack position and the target nut region, determine whether the plate edge defect is a plate edge crack.

[0082] For example, Figure 2 For example, the current riveted plate edge area to be identified is 201, and the riveted point nut area closest to 201 is 202, which is the target nut area.

[0083] In the embodiment of the present application, the plate edge defect can be determined based on whether the nut near the plate edge area is a crack. Considering that when a crack appears in the plate edge area, a nut crack usually appears in the nearby nut area, if the first abnormal result is not output, it means that there is no nut crack in the image to be detected, and then it can be determined that the plate edge defect is not a plate edge crack, but an interference defect such as scratches, vinyl, handwriting, etc. Similarly, if the first abnormal result is output and there is a nut crack in the image to be detected, the plate edge defect can be classified based on the positional relationship between the nut crack position and the target nut area, that is, whether the nut crack position is located in the target nut area closest to the riveted plate edge area, so as to solve the difficulty of defect type classification and achieve high accuracy.

[0084] In some embodiments of the present application, in order to accurately determine whether the plate edge defect is a plate edge crack, Figure 7 is a flowchart of a riveting defect identification method provided by another embodiment of the present application. Step 630 may include Figure 7 Steps 710 - 740 are shown.

[0085] Step 710, when the nut crack position is outside the target nut area, determining that the plate edge defect is not a plate edge crack;

[0086] Step 720, when the nut crack position is located within the target nut area, based on the preset pixel interval corresponding to the plate edge crack, the pixel values ​​of all pixels in the riveted plate edge area in the image to be inspected are adjusted;

[0087] Step 730, performing contour detection on the defect position in the riveted plate edge region in the adjusted image to be detected to obtain a contour region;

[0088] Step 740: When the contour area satisfies a preset crack morphology condition, determine that the plate edge defect is a plate edge crack.

[0089] In the embodiment of the present application, if the position of the nut crack in the image to be detected is outside the target nut area, it means that the position of the nut crack is far from the board edge area, and the nut closest to the board edge area has no crack, so it can be directly determined that the defect in the board edge area is an interference defect, not a real crack. If the position of the nut crack is within the target nut area, that is, the nut closest to the board edge area has a crack, it can be determined that the board edge defect may be a crack. On this basis, the two prior knowledge of board edge cracks, pixels and morphology, can be further combined to further effectively distinguish the board edge defects, improve the complexity of the judgment process, and then screen out non-crack defects such as black glue, cracked text and handwriting in the riveting process, and improve the recognition accuracy of board edge defects.

[0090] In some embodiments of the present application, the above step 720 adjusts the pixel values ​​of all pixels in the riveted plate edge area in the image to be detected, which may include the following steps:

[0091] Obtain a pixel histogram of the image to be detected;

[0092] Based on the pixel histogram, the pixel values ​​of all pixels in the riveted plate edge area are determined;

[0093] Determine a pixel point whose pixel value is within a preset pixel interval as a first pixel point, and a pixel point other than the first pixel point as a second pixel point;

[0094] The pixel value of the first pixel point is adjusted to a first preset pixel value, and the pixel value of the second pixel point is adjusted to a second preset pixel value.

[0095] Specifically, the purpose of adjusting the pixel value is to quickly identify the defect contour from the riveted plate edge area. The preset pixel interval is the pixel interval corresponding to the real plate edge crack. Therefore, the preset pixel interval is used to screen the pixel points corresponding to the defect position in the riveted plate edge area, that is, the first pixel point is the pixel point corresponding to the defect position, and the second pixel point is the pixel point corresponding to the non-defect position. The preset pixel threshold, the first preset pixel value, and the second preset pixel value can be set according to specific needs, and this application does not make specific limitations on this.

[0096] As an example, the preset pixel interval can be set to 0 to 50, the first preset pixel value can be set to 0, and the second preset pixel value can be set to 255. Then, the first pixel point in the riveted plate edge area whose pixel value is in the range of 0 to 50 can be reassigned to 0, and the second pixel point outside the range of 0 to 50 can be reassigned to 255.

[0097] It should be noted that the preset pixel threshold, the first preset pixel value and the second preset pixel value may also be set to other values, for example, the preset pixel threshold is set to 60, the first preset pixel value is set to 2, and the second preset pixel value is set to 260.

[0098] In the embodiment of the present application, by reassigning the first pixel point and the second pixel point, the difference in pixel values ​​between the defect position and the background area can be increased. Contour recognition is performed on this basis, which can reduce the difficulty of contour recognition and thus improve the accuracy of contour recognition, and quickly identify the defect contour in the riveted plate edge area.

[0099] In some embodiments of the present application, the image to be detected includes a plurality of riveted plate edge regions, and the preset crack morphology condition in the above step 740 may include at least one of the following:

[0100] The ratio of the perimeter of the contour area in the riveted plate edge area to the target length is greater than a first preset ratio threshold; the ratio of the area of ​​the contour area in the riveted plate edge area to the target area is greater than a second preset ratio threshold.

[0101] The target length is the length of the longest side of the riveted plate edge region where the contour region is located, and the target area is the area of ​​the riveted plate edge region where the contour region is located.

[0102] The first preset ratio and the second preset ratio can be set according to specific needs, and this application does not make any specific limitation on this.

[0103] As an example, the first preset ratio can be 1, and the second preset ratio can be 1 / 10. When the ratio of the contour circumference to the target length is less than 1, or the ratio of the contour area to the target area is less than 0.1, it means that the morphology of the plate edge defect does not match the real crack. Otherwise, the plate edge defect is a real crack.

[0104] In the embodiments of the present application, when a crack occurs on the edge of the plate, the crack position usually extends to the end, and the inventors have found through many experiments that the perimeter of the crack area on the edge of the plate usually exceeds the length of the longest side of the riveted plate edge area, and the area of ​​the crack area on the edge of the plate usually accounts for more than a certain proportion of the area of ​​the riveted plate edge area. Therefore, combined with this prior knowledge, the preset crack morphology conditions can be used to determine the contour of the plate edge defect in morphology, and the plate edge defects can be accurately identified and classified based on whether the defect morphology matches the actual crack morphology.

[0105] For example, Figure 8 As shown, Example 1 and Example 2 correspond to two riveted plate edge regions respectively, and the two riveted plate edge regions can be different riveted plate edge regions from the same image to be detected, or from different images to be detected. After obtaining the pixel histograms of the two riveted plate edge regions, the pixel values ​​of all pixels in the region can be clearly understood, and then the pixel values ​​are reassigned and contour detection is performed, and finally the contour region of the defect position in the riveted plate edge region is obtained.

[0106] In some embodiments of the present application, in order to solve the difficulty of identifying riveting defects under dark light conditions in a workshop, the collected image can be equalized. After obtaining the shock tower riveted image marked as defect-free in the riveted area in step 110, the method can also include: performing image equalization processing on the shock tower riveted image; step 120 uses the shock tower riveted image to train an unsupervised classification model, which can include: training the unsupervised classification model using the shock tower riveted image after image equalization processing.

[0107] Among them, the image equalization technology can be used to perform image equalization processing on the shock tower riveting image. The image equalization technology can include but is not limited to histogram equalization, sharpening and other operations.

[0108] It should be noted that, in step 130, the image to be detected may be subjected to image equalization processing, and the image to be detected after the image equalization processing may be input into the riveting defect recognition model.

[0109] As an example, Fig. 9 As shown, after performing image equalization processing on the left image 901, an equalized image 902 on the right can be obtained.

[0110] In an embodiment of the present application, if an image of the riveting of a shock tower is generated by shooting in a dark light scene in a workshop, image equalization processing can be performed on it to reduce the impact of the dark light conditions in the workshop on the image quality, thereby reducing the negative impact of the dark light conditions in the workshop on the recognition accuracy of the model, thereby ensuring the accuracy of riveting defect recognition in dark light.

[0111] It is understandable that the rivet defect recognition method provided in the embodiment of the present application can be executed by an electronic device or a control module in a rivet defect recognition device for executing the rivet defect recognition method. The rivet defect recognition device is described in detail below.

[0112] Fig.10 Schematic diagram of the structure of a riveting defect identification device provided in an embodiment of the present application. Fig.10 As shown, the riveting defect identification device 1000 may include: an acquisition module 1010 , a model training module 1020 , an input module 1030 and a defect identification module 1040 .

[0113] Among them, the acquisition module 1010 is used to obtain the riveted image of the shock tower marked as defect-free in the riveted area; the model training module 1020 is used to use the unsupervised classification model as the basic network, and use the riveted image of the shock tower to train the unsupervised classification model to obtain a trained riveted defect recognition model; the input module 1030 is used to input the image to be detected into the riveted defect recognition model when the image to be detected is acquired, so that the riveted defect recognition model can perform defect recognition on the riveted area of ​​the image to be detected; the defect recognition module 1040 is used to determine that there is a riveted defect in the riveted area in the image to be detected when the riveted defect recognition model outputs an abnormal recognition result.

[0114] The riveting defect recognition device provided by the present application obtains the riveted shock tower image marked as defect-free in the riveted area. On this basis, the unsupervised classification model is used as the basic network, and the unsupervised classification model is trained using the riveted shock tower image to obtain a trained riveted defect recognition model. In this way, when the image to be detected is obtained, the image to be detected is directly input into the riveted defect recognition model, and the riveted defect recognition model can perform defect recognition on the riveted area of ​​the image to be detected. When the riveted defect recognition model outputs an abnormal recognition result, it is determined that there is a riveted defect in the riveted area of ​​the image to be detected. Therefore, the riveted defect recognition method only needs to be trained with the riveted shock tower image without defects, and the training task can be completed without defective samples. The performance is better, and the defects in the riveted area in the image to be detected can be effectively identified to complete the riveting quality inspection, which can be applied to actual industrial scenarios.

[0115] In some embodiments of the present application, the riveted area includes a riveted point nut area and a riveted plate edge area, and the device also includes: a division module, which is used to divide the shock tower riveted image into a first image without defects in the riveted point nut area and a second image without defects in the riveted plate edge area after acquiring the shock tower riveted image marked as defect-free in the riveted area; the model training module includes: a training unit, which is used to train the unsupervised classification model using the first image and the second image.

[0116] In some embodiments of the present application, the unsupervised classification model includes a first classification model and a second classification model, the riveting defect recognition model includes a first classifier and a second classifier, and the training unit is specifically used to: use the first image as a training sample of the first classification model to train the first classification model to obtain a first classifier, wherein the first classifier is used to identify nut cracks in the nut area of ​​the riveted point in the image to be detected; use the second image as a training sample of the second classification model to train the second classification model to obtain a second classifier, wherein the second classifier is used to identify plate edge defects in the riveted plate edge area in the image to be detected.

[0117] In some embodiments of the present application, the abnormal recognition result includes a first abnormal result and / or a second abnormal result, and the defect recognition module 1040 is specifically used to: when outputting the first abnormal result, determine that a nut crack exists in the nut area of ​​the riveted point in the image to be detected; when outputting the second abnormal result, determine that a plate edge defect exists in the riveted plate edge area in the image to be detected.

[0118] In some embodiments of the present application, the image to be detected includes multiple riveted point nut areas, and the first abnormal result is used to identify the position of the nut crack in the riveted point nut area. The device also includes: a defect recognition module 1040, which is also used to determine that the plate edge defect is not a plate edge crack after determining that there is a plate edge defect in the riveted plate edge area in the image to be detected without outputting the first abnormal result; a screening module, which is used to screen a target nut area that is closest to the riveted plate edge area from multiple riveted point nut areas when the first abnormal result is output; the defect recognition module 1040 is also used to determine whether the plate edge defect is a plate edge crack based on the positional relationship between the nut crack position and the target nut area.

[0119] In some embodiments of the present application, the defect identification module 1040 includes: a determination unit, which is used to determine that the plate edge defect is not a plate edge crack when the nut crack position is outside the target nut area; an adjustment unit, which is used to adjust the pixel values ​​of all pixel points in the riveted plate edge area in the image to be detected based on a preset pixel interval corresponding to the plate edge crack when the nut crack position is within the target nut area; a detection unit, which is used to perform contour detection on the defect position in the riveted plate edge area in the adjusted image to be detected to obtain a contour area; the determination unit is also used to determine that the plate edge defect is a plate edge crack when the contour area meets a preset crack morphology condition.

[0120] In some embodiments of the present application, the adjustment unit is specifically used to: obtain a pixel histogram of the image to be detected; determine the pixel values ​​of all pixel points in the riveted plate edge area based on the pixel histogram; determine the pixel points whose pixel values ​​are located in a preset pixel interval as first pixel points, and the pixel points other than the first pixel points as second pixel points; adjust the pixel value of the first pixel point to a first preset pixel value, and adjust the pixel value of the second pixel point to a second preset pixel value.

[0121] In some embodiments of the present application, the image to be detected includes multiple riveted plate edge areas, and the preset crack morphology conditions include at least one of the following: the ratio of the circumference of the contour area within the riveted plate edge area to the target length is greater than a first preset ratio threshold; the ratio of the area of ​​the contour area within the riveted plate edge area to the target area is greater than a second preset ratio threshold; wherein the target length is the length of the longest side of the riveted plate edge area where the contour area is located, and the target area is the area of ​​the riveted plate edge area where the contour area is located.

[0122] In some embodiments of the present application, the device also includes: an equalization module, which is used to perform image equalization processing on the shock tower riveting image after obtaining the shock tower riveting image marked as defect-free in the riveting area; the model training module is specifically used to: use the shock tower riveting image after image equalization processing to train the unsupervised classification model.

[0123] The riveting defect identification device provided in the embodiment of the present application can achieve Figure 1-Figure 9 The various processes implemented by the electronic device in the method embodiment can achieve the same technical effect, and to avoid repetition, they will not be described here.

[0124] Fig.11 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application.

[0125] like Fig.11 As shown, the electronic device 1100 in this embodiment may include a processor 1101 and a memory 1102 storing computer program instructions.

[0126] Specifically, the processor 1101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0127] The memory 1102 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 1102 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 1102 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 1102 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 1102 is a non-volatile solid-state memory. The memory may include a read-only memory (ROM), a random access memory (RAM), a disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical, or other physical / tangible memory storage device. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) having software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to the embodiments of the present application.

[0128] The processor 1101 implements any one of the riveting defect identification methods in the above embodiments by reading and executing computer program instructions stored in the memory 1102 .

[0129] In one example, the electronic device 1100 may further include a communication interface 1103 and a bus 1110. Fig.11 As shown, the processor 1101, the memory 1102, and the communication interface 1103 are connected via a bus 1110 and communicate with each other.

[0130] The communication interface 1103 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0131] Bus 1110 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 1110 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0132] The electronic device provided in the embodiment of the present application can realize Figure 1-Figure 9 The various processes implemented by the electronic device in the method embodiment can achieve the same technical effect, and to avoid repetition, they will not be described here.

[0133] In combination with the riveting defect identification method in the above embodiment, the embodiment of the present application can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, the steps of any one of the riveting defect identification methods in the above embodiment are implemented.

[0134] In combination with the riveting defect identification method in the above embodiment, the embodiment of the present application can provide a computer program product for implementation. The (computer) program product is stored in a non-volatile storage medium, and when the program product is executed by at least one processor, the steps of any one of the riveting defect identification methods in the above embodiment are implemented.

[0135] The embodiment of the present application further provides a chip, which includes a processor and a communication interface, the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned riveting defect identification method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0136] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0137] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0138] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), appropriate firmware, plug-in, function card, etc. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0139] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0140] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0141] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A riveting defect identification method, characterized in that: The method comprises: Acquire an image of the shock tower riveted joint with the riveted area marked as free of defects; The unsupervised classification model is used as a basic network, and the shock tower riveting image is used to train the unsupervised classification model to obtain a trained riveting defect recognition model; When the image to be detected is acquired, the image to be detected is input into the riveting defect recognition model, so that the riveting defect recognition model performs defect recognition on the riveting area of ​​the image to be detected; In the case where the riveting defect recognition model outputs an abnormal recognition result, it is determined that a riveting defect exists in the riveting area in the image to be detected.

2. The method according to claim 1, characterized in that The riveted area includes a riveted point nut area and a riveted plate edge area. After acquiring the riveted image of the shock tower marked as defect-free in the riveted area, the method further includes: Dividing the shock tower riveting image into a first image in which the riveted point nut area is free of defects and a second image in which the riveted plate edge area is free of defects; The step of training the unsupervised classification model using the shock tower riveting image includes: The unsupervised classification model is trained using the first image and the second image.

3. The method according to claim 2, characterized in that The unsupervised classification model includes a first classification model and a second classification model, the riveting defect recognition model includes a first classifier and a second classifier, and the training of the unsupervised classification model using the shock tower riveting image includes: Using the first image as a training sample of the first classification model to train the first classification model to obtain the first classifier, wherein the first classifier is used to identify nut cracks in a nut area of ​​a riveted point in the image to be detected; The second classification model is trained by using the second image as a training sample of the second classification model to obtain the second classifier, wherein the second classifier is used to identify the plate edge defects in the riveted plate edge area in the image to be detected.

4. The method according to claim 2 or 3, characterized in that: The abnormal recognition result includes a first abnormal result and / or a second abnormal result. When the riveting defect recognition model outputs the abnormal recognition result, determining that there is a riveting defect in the riveting area in the image to be detected includes: In the case where the first abnormal result is output, determining that there is a nut crack in the nut area of ​​the riveted point in the image to be detected; When the second abnormal result is output, it is determined that a plate edge defect exists in the riveted plate edge region in the image to be detected.

5. The method according to claim 4, characterized in that The image to be detected includes a plurality of riveted point and nut regions, the first abnormal result is used to identify the position of the nut crack in the riveted point and nut region, and after determining that there is a plate edge defect in the riveted plate edge region in the image to be detected, the method further includes: In the case where the first abnormal result is not output, determining that the plate edge defect is not a plate edge crack; In the case of outputting the first abnormal result, selecting a target nut area closest to the riveted plate edge area from the plurality of riveted point nut areas; Based on the positional relationship between the nut crack position and the target nut area, it is determined whether the plate edge defect is a plate edge crack.

6. The method according to claim 5, characterized in that The determining whether the plate edge defect is a plate edge crack based on the positional relationship between the nut crack position and the target nut region includes: In the case where the nut crack position is outside the target nut area, determining that the plate edge defect is not a plate edge crack; When the nut crack position is located within the target nut area, adjusting the pixel values ​​of all pixels in the riveted plate edge area in the image to be detected based on a preset pixel interval corresponding to the plate edge crack; Performing contour detection on the defect position in the riveted plate edge region in the adjusted image to be detected to obtain a contour region; When the preset crack morphology condition is satisfied in the contour area, the plate edge defect is determined to be a plate edge crack.

7. The method according to claim 6, characterized in that The adjusting the pixel values ​​of all pixels in the riveted plate edge region in the image to be detected based on the preset pixel interval corresponding to the plate edge crack comprises: Acquire a pixel histogram of the image to be detected; Based on the pixel histogram, determining the pixel values ​​of all pixels in the riveted plate edge area; Determine a pixel point whose pixel value is within the preset pixel interval as a first pixel point, and a pixel point other than the first pixel point as a second pixel point; The pixel value of the first pixel point is adjusted to a first preset pixel value, and the pixel value of the second pixel point is adjusted to a second preset pixel value.

8. The method according to claim 6, characterized in that The image to be detected includes a plurality of riveted plate edge regions, and the preset crack morphology condition includes at least one of the following: The ratio of the perimeter of the contour area within the riveted plate edge area to the target length is greater than a first preset ratio threshold; The ratio of the area of ​​the contour area in the riveted plate edge area to the target area is greater than a second preset ratio threshold; The target length is the length of the longest side of the riveted plate edge region where the contour region is located, and the target area is the area of ​​the riveted plate edge region where the contour region is located.

9. A riveting defect identification device, characterized in that: The device comprises: An acquisition module, for acquiring a riveted image of a shock tower marked as defect-free in a riveted area; A model training module, used to use the unsupervised classification model as a basic network, and train the unsupervised classification model using the shock tower riveting image to obtain a trained riveting defect recognition model; An input module, used for inputting the image to be detected into the riveting defect recognition model when the image to be detected is acquired, so that the riveting defect recognition model performs defect recognition on the riveting area of ​​the image to be detected; The defect recognition module is used to determine whether a riveting defect exists in the riveting area in the image to be detected when the riveting defect recognition model outputs an abnormal recognition result.

10. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, the steps of the riveting defect identification method according to any one of claims 1 to 8 are implemented.

Images