A vehicle door appearance detection method based on machine vision

Through the machine vision-based door appearance detection method, the use of neural networks to detect welding defects and integrity, the problem of low manual detection efficiency is solved, automated and efficient door welding inspection is realized, and production efficiency and quality management are improved.

CN112634236BActive Publication Date: 2024-11-01GUANGZHOU ZSROBOT INTELLIGENT EQUIP CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202011556100.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-12-24
Publication Date
2024-11-01
Estimated Expiration
2040-12-24

AI Technical Summary

Technical Problem

The prior art has problems of slow manual visual inspection speed, missed inspection, and missed inspection in automotive door welding inspection, which cannot meet the needs of fast and efficient inspection of mass production.

Method used

The door appearance detection method based on machine vision is adopted, welding images are obtained through industrial cameras, filtering and feature extraction are performed, and welding defects and integrity detection is carried out, and threshold determination and grading methods are used to achieve automated detection.

Benefits of technology

Significantly reduce the labor intensity of inspection personnel, avoid mis-inspection and missed inspection, improve door production efficiency, shorten production cycles, save costs, and realize modern management of door quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112634236B_ABST
    Figure CN112634236B_ABST
Patent Text Reader

Abstract

The present invention provides a vehicle door appearance detection method based on machine vision, which belongs to the field of appearance detection methods. The vehicle door welding appearance image after filtering is copied, and the images before and after the copy are set as image 1 and image 2; the image 1 is scaled and input into a neural network A, the image defect confidence is output and compared with a set threshold value A, the welding defects are counted and marked, and the welding defects are graded A to obtain a welding defect grading result; the image 2 is converted into a color channel, and then segmented to obtain an image block, the welding area features are extracted and normalized into color features, the color features are input into a neural network B, the image non-welding probability is output and compared with a set threshold value B, the non-welding area is counted and marked, and the welding integrity grading result is obtained. The vehicle door appearance image is subjected to basic processing and target detection, and welding defect and welding integrity analysis results are obtained, so that the detection becomes faster and more efficient.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of appearance detection methods, and in particular to a vehicle door appearance detection method based on machine vision. Background Art

[0002] In the welding process of automobile doors, various uncertain factors may cause welding leaks and uneven welding of the doors. In the inspection of door welding, manual visual inspection is slow, prone to misdetection and missed detection, low efficiency, and cannot meet production requirements. In the batch door welding appearance inspection, it is necessary to quickly and efficiently detect and judge welding defects and welding integrity in order to shorten the automobile production cycle. In order to solve the drawbacks of manual visual inspection methods, a new detection method is needed, which combines modern machine vision technology to detect and analyze the appearance of the inspected door and trace the problem analysis. Summary of the invention

[0003] In view of the shortcomings of the prior art, the present invention proposes a vehicle door appearance detection method based on machine vision, which processes a series of vehicle door appearance images and inputs them into a neural network established in advance, determines welding defects and welding integrity through thresholds, and then uses a grading method to present the grading results of its appearance detection, so as to realize the automation and intelligence of vehicle door appearance detection.

[0004] The technical solution of the present invention is achieved in this way:

[0005] Step 1: using an industrial camera to obtain an appearance image of the door welding, and filtering the appearance image;

[0006] Step 2: Assume that the appearance image after filtering is image 1, copy image 1 to obtain image 2, process image 1 using step 21, and process image 2 using step 22:

[0007] Step 21, scaling the image 1;

[0008] Step 22, performing color channel conversion on image 2, and then obtaining image blocks through global image segmentation, extracting welding area features and normalizing them into color features;

[0009] Step 3, inputting the scaled image in step 21 into the neural network A, the neural network A outputs the image defect confidence, compares the defect confidence with the set threshold A, counts and marks the welding defects, and classifies the area and number of the welding defects A to obtain the appearance welding defect classification result;

[0010] Step 4: Input the color features described in step 22 into the neural network B, the neural network B outputs the probability of unwelded images, compares the unwelded probability with the set threshold B, and counts and marks the unwelded areas, and grades the areas of the unwelded areas B to obtain the appearance welding integrity grading results.

[0011] In step 1, the industrial camera is a CCD camera, and the door appearance image is an RGB image.

[0012] In step 1, the filtering preprocessing adopts 3x3 median filtering method.

[0013] In step 22, the color channel conversion is to convert the RGB color space into the HSV color space, where R in the RGB color space represents red, G represents green, and B represents blue, and H in the HSV color space represents color, S represents depth, and V represents brightness.

[0014] In step 22, the global image segmentation adopts triangle automatic global threshold segmentation.

[0015] In step 22, the welding area features are extracted using third-order color moments.

[0016] In step 3, the model of the neural network A adopts yolov3.

[0017] In step 3, in the classification A, the area marked as the welding defect area is set as S0, the defect area detection area is set to [S1, S2], the defect number threshold N is set, and the welding defect area is traversed:

[0018] Case 31: If S0>S2, it is classified as failing grade;

[0019] Case 32: If S0∈[S1, S2], and its number is greater than N, it is classified as a failing grade;

[0020] Case 33, different from Case 1 and Case 2, is classified as a passing grade.

[0021] In step 4, in the classification B, the area A0 marked as the unwelded area is set, the unwelded area detection threshold A1 is set, and the unwelded area is traversed: if A0>A1, it is classified as a failing grade, and the rest are classified as a passing grade.

[0022] The appearance welding defect grading result and the appearance welding integrity grading result are analyzed. If both are qualified, the assembly line controls the car door to flow into the next production line. Otherwise, the assembly line controls the car door to flow into the recycling line.

[0023] The beneficial effects of the present invention are:

[0024] The vehicle door appearance detection method based on machine vision provided by the present invention uses a machine vision method to detect the appearance of vehicle door welding. On the one hand, it greatly reduces the labor intensity of detection personnel and avoids the problems of false detection and missed detection in manual detection. On the other hand, it speeds up the production time of vehicle doors, saves production costs, and improves the production efficiency of vehicle doors. In addition, the storage function of the computer can be used to easily track a series of parameters such as vehicle door product quality and welding yield, thereby realizing modern management of vehicle door production. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0026] Figure 1 It is a flow chart of a vehicle door appearance detection method based on machine vision provided in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] like Figure 1 As shown, a vehicle door appearance detection method based on machine vision provided in this embodiment includes the following steps:

[0029] Step 1: Use an industrial camera to obtain the appearance image of the door welding, and perform filtering on the appearance image.

[0030] The industrial camera used is a CCD camera. The camera parameters are set when taking pictures until the image is transmitted to the computer and a clear image of the door can be displayed. The image collected by the industrial camera is an RGB image.

[0031] The filter preprocessing uses the 3x3 median filter method to eliminate image noise, sort the surrounding pixels and the central pixel, and then take the median value;

[0032] Step 2: Let the appearance image after filtering be image 1, copy image 1 to obtain image 2, process image 1 using step 21, and process image 2 using step 22:

[0033] Step 21, scaling the image 1;

[0034] Scale the image to 448x448 pixels and increase its clarity and smoothness;

[0035] Step 22, performing color channel conversion on image 2, and then obtaining image blocks through global image segmentation, extracting welding area features and normalizing them into color features;

[0036] Color channel conversion is to convert RGB color space to HSV color space. R in RGB color space represents red, G represents green, and B represents blue. In HSV color space, H represents color, S represents depth, and V represents brightness. HSV color space is conducive to color segmentation. H and S components are used to represent color distance. Color distance refers to the numerical difference between two colors. For different color areas, H and S variables are mixed to prepare for image segmentation.

[0037] The global image segmentation adopts triangle automatic global threshold segmentation, uses global information to find the optimal segmentation threshold for the whole image, and segments the image according to the threshold to obtain several image blocks;

[0038] The welding area features are extracted using the third-order color moment, which transforms the features of the image block into mathematical information. Any color distribution in the image can be represented by the color moment. The color distribution information is mainly concentrated in the low-order moment. The third-order color moment can satisfy the color distribution expression of the image. Each image block obtains nine components, which contain three color components. Each color component contains three low-order moments. The third-order color moment does not need to vectorize the features, and the color features are relatively concise.

[0039] Step 3: Input the scaled image in step 21 into neural network A. Neural network A outputs the image defect confidence, compares the defect confidence with the set threshold A, counts and marks the welding defects, and classifies the area and number of welding defects A to obtain the appearance welding defect grading result.

[0040] The model of neural network A adopts yolov3, and the door samples collected in the experiment are used as the training set to obtain the model, and the prediction results with defect confidence above the set threshold A are marked as defects;

[0041] In classification A, the area marked as welding defect area is set as S0, the defect area detection area is set as [S1, S2], the defect number threshold N is set, and the welding defect area is traversed:

[0042] Case 31: If S0>S2, it is classified as failing grade;

[0043] Case 32: If S0∈[S1, S2], and its number is greater than N, it is classified as a failing grade;

[0044] Situation 33, unlike situations 1 and 2, is classified as a passing grade;

[0045] All statistically and marked defective areas are inspected. If the area of ​​the welding defective area is greater than the set threshold, it is classified as a failure level. If the area of ​​the welding defective area reaches the threshold interval and the number reaches the set value, it is also classified as a failure level. The rest are classified as passing levels.

[0046] Step 4: Input the color features of step 22 into neural network B. Neural network B outputs the probability of unwelded image, compares the probability of unwelded image with the set threshold value B, and counts and marks the unwelded area. The area of ​​the unwelded area is graded B to obtain the appearance welding integrity grading result.

[0047] Neural network B is a neural network classifier, which can be directly used by the pre-trained model in the experiment, or can be trained by itself in production. The prediction results with the probability of non-welding above the set threshold B are marked as non-welding areas;

[0048] In classification B, the area A0 marked as the unwelded area is set, the unwelded area detection threshold A1 is set, and the unwelded area is traversed: if A0>A1, it is classified as a failing grade, and the rest are classified as a passing grade;

[0049] The analyzed appearance welding defect grading results and appearance welding integrity grading results, if both are qualified, the assembly line controls the car door to flow into the next production line, otherwise, the assembly line controls the car door to flow into the recycling line. In the computer system, the car door welding defects or welding integrity of the unqualified level can also be tracked and analyzed, the defective products can be quickly repaired, and it is convenient to establish defective product data analysis, improve process methods, and enhance the automation and intelligence of car door detection.

[0050] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A vehicle door appearance detection method based on machine vision, characterized in that: Step 1: using an industrial camera to obtain an appearance image of the door welding, and filtering the appearance image; Step 2: Assume that the appearance image after filtering is image 1, copy image 1 to obtain image 2, process image 1 using step 21, and process image 2 using step 22: Step 21, scaling the image 1; Step 22, performing color channel conversion on image 2, and then obtaining image blocks through global image segmentation, extracting welding area features and normalizing them into color features; Step 3: Input the scaled image in step 21 into the neural network A, the neural network A outputs the image defect confidence, compares the defect confidence with the set threshold A, counts and marks the welding defects, and classifies the area and number of the welding defects A to obtain the appearance welding defect classification result, wherein in the classification A, the area marked as the welding defect area is set as S0, the defect area detection area is set [S1, S2], the defect number threshold N is set, and the welding defect area is traversed: Case 31: If S0>S2, it is classified as failing grade; Case 32: If S0∈[S1, S2], and its number is greater than N, it is classified as failing grade; Situation 33, unlike situations 1 and 2, is classified as a passing grade; Step 4, input the color features described in step 22 into the neural network B, the neural network B outputs the probability of unwelded image, compares the unwelded probability with the set threshold B, and counts and marks the unwelded area, and classifies the area of ​​the unwelded area into a grade B to obtain the appearance welding integrity grading result, wherein, in the grade B, the area A0 marked as the unwelded area is set, the unwelded area detection threshold A1 is set, and the unwelded area is traversed: if A0>A1, it is classified as a failing grade, and the rest are classified as a passing grade.

2. The vehicle door appearance detection method based on machine vision according to claim 1, characterized in that: In step 1, the industrial camera is a CCD camera, and the door appearance image is an RGB image.

3. The vehicle door appearance detection method based on machine vision according to claim 1, characterized in that: In step 1, the filtering preprocessing adopts 3x3 median filtering method.

4. The vehicle door appearance detection method based on machine vision according to claim 1, characterized in that: In step 22, the color channel conversion is to convert the RGB color space into the HSV color space. The R in the space represents red, G represents green, and B represents blue. In the HSV color space, H represents color, S represents depth, and V represents brightness.

5. The vehicle door appearance detection method based on machine vision according to claim 1, characterized in that: In step 22, the global image segmentation adopts triangle automatic global threshold segmentation.

6. The vehicle door appearance detection method based on machine vision according to claim 1, characterized in that: In step 22, the welding area features are extracted using third-order color moments.

7. The vehicle door appearance detection method based on machine vision according to claim 1, characterized in that: In step 3, the model of the neural network A adopts yolov3.

8. The vehicle door appearance detection method based on machine vision according to claim 1, characterized in that: The appearance welding defect grading result and the appearance welding integrity grading result are analyzed. If both are qualified, the assembly line controls the vehicle door to flow into the next production line; otherwise, the assembly line controls the vehicle door to flow into the recycling line.