Online visual detection system and detection method for automobile seat surface defects based on deep learning

Through the deep learning-based automotive seat surface defect detection system, the multi-layer convolutional neural network and 3D morphological path planning, combined with the 4-zone light source design, the problem of difficult to identify variable seat surface defects is solved in traditional detection, and the problem of traditional detection is realized. Efficient and full coverage detection effect.

CN119086555AInactive Publication Date: 2024-12-06JILIN CHANGGUANG JINGYI INTELLIGENT TECHNOLOGY CO LTD

Patent Information

Application Number
CN202411076985.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

Traditional algorithms are difficult to identify surface defects of car seats, especially due to the variable shape, material and color of seats, which leads to low detection efficiency and artificial visual inspections are prone to fatigue, making full-coverage detection impossible.

Method used

Adopting a deep learning-based online visual detection system for surface defects of the car seat, including robots, telescopic motion mechanisms, lighting and imaging systems, and electronically controlled image processing systems, defect detection models are constructed through multi-layer convolutional neural networks, combining 3D morphological paths. Planning and 4-zone light source design to achieve full coverage detection.

Benefits of technology

It improves detection efficiency and accuracy, can effectively identify defects on the seat surface, make up for the shortcomings of traditional visual detection algorithms, and achieves a full-coverage detection effect.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119086555A_ABST
    Figure CN119086555A_ABST
Patent Text Reader

Abstract

The invention relates to an automobile seat surface defect online visual detection system and detection method based on deep learning. The detection system comprises a robot A, a robot illumination and imaging system A, a telescopic movement mechanism A and a telescopic illumination and imaging system A, the system comprises a robot A, a robot B, a robot lighting and imaging system B, a telescopic movement mechanism B and a telescopic lighting and imaging system B; and a seat posture mistake proofing mechanism, an electric control and image processing system and a display. According to the automobile seat surface defect online visual detection system based on deep learning, through hardware joint design of the optical imaging system and the electric control system, the problem that a traditional automobile seat detection system cannot cover the whole seat surface under a vertical shooting visual angle is solved; the optimal vertical full-coverage detection visual field path planning can be made according to different seat shapes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of optical detection technology, and in particular to an online visual detection system and method for surface defects of automobile seats based on deep learning. Background Art

[0002] As an important part of the whole vehicle, car seats are prone to some minor surface defects during the production process, such as oil stains, scratches, wrinkles, thread ends, etc., which will seriously affect the competitiveness of the product when it is delivered.

[0003] Machine vision inspection technology is mainly used in the field of automotive parts for dimension measurement, mis-assembly detection, etc. However, in the detection of surface defects of automotive seats, surface defects have the characteristics of random positions and no fixed abnormal features. At the same time, the surface shape, material, color and features of automotive seats are variable, making it difficult for traditional algorithms to identify. At present, manual visual inspection is still used, which is easy to fatigue, has vague standards and high management costs.

[0004] To overcome the shortcomings of the existing technology, Wang Han from Guangdong University of Technology filed a patent application with application number CN105806849A, titled "Automotive Seat Surface Defect Detection System and Detection Method Based on Machine Vision". The system includes a light source device, a single-chip numerical control device, an image acquisition device, an image processing and analysis platform, a database system, and a hardware device. This method requires the installation of multiple static cameras in all directions of the car seat, which is not flexible enough for different models of seats, and it is impossible to capture the detection image at the best position according to the seat shape, which affects the detection effect.

[0005] Luo Weian from Guangdong University of Technology filed a patent application with application number CN106226325A, titled A seat surface defect detection system and method based on machine vision. It includes a robotic arm, a numerical control device, a laser rangefinder, an image acquisition device, an image processing and analysis platform, and a database system. It combines traditional visual algorithms such as color recognition and template matching, and systematically provides a relatively complete automotive seat surface defect detection system and solution from hardware architecture to software algorithm. This solution detects seat defects by comparing the detection image with template data. There are many types of shapes, colors, and materials of automotive seats, and the efficiency of multi-template matching of the detection image is low. At the same time, the solution only includes a camera installed on the robotic arm. Due to the limited spatial path, the field of view cannot fully cover the seat surface. This method is suitable for scenes with fewer types, low timeliness requirements, and detection of local areas of the seat.

[0006] Invention patent CN112232399B discloses a car seat defect detection method based on multi-feature fusion machine learning. By extracting the color and texture features of multi-category car seat materials, the multi-feature fusion classifier is trained to identify material category information and screen out material category anomalies; based on the classification results, defect detection is performed in combination with Blob analysis to determine whether there is damage or stains. The technical feature is that it integrates multi-dimensional features of the image, does not require multi-template matching, uses machine learning to train the classifier, automatically classifies car seat material categories, classifies materials of different colors and materials, and screens out abnormal defects in material types; the matching efficiency is high and the timeliness is high. The disadvantage of this method is that its processing method is still based on image preprocessing and image fusion of traditional algorithms. Due to the variable texture of car seat defect features, the irregular defect location and features, it is impossible to manually match all situations, and the traditional image algorithm is not effective. Summary of the invention

[0007] The present invention aims to solve the technical problems in the prior art and provide an online visual inspection system and method for automobile seat surface defects based on deep learning.

[0008] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0009] An online visual inspection system for automobile seat surface defects based on deep learning, comprising:

[0010] Robot A, robot lighting and imaging system A, telescopic motion mechanism A, telescopic lighting and imaging system A;

[0011] Robot B, robot lighting and imaging system B, telescopic motion mechanism B, telescopic lighting and imaging system B; and

[0012] Seat posture error prevention mechanism, electronic control and image processing system and display; the display is used to display the detection process and results; the electronic control and image processing system is used to send a start detection signal to control robot A and / or robot B to move according to the preset point position;

[0013] Robot A and robot B are respectively arranged before and after the seat inspection station, and robot A and robot B are respectively connected to robot lighting and imaging system A and robot lighting and imaging system B; robot lighting and imaging system A and / or robot lighting and imaging system B are respectively used to photograph the front, back and top of the inspected seat along a preset path;

[0014] A telescopic motion mechanism A and a telescopic motion mechanism B are respectively provided on both sides of the seat inspection station, and the telescopic motion mechanism A and the telescopic motion mechanism B are respectively connected to the inspection slide rails, and a telescopic lighting and imaging system A and a telescopic lighting and imaging system B are respectively provided on each inspection slide rail;

[0015] When the seat to be tested enters the measuring station, the seat posture error prevention mechanism is used to screen the seat posture to be tested;

[0016] When the inspected seat enters the measuring station, the telescopic motion mechanism A and / or the telescopic motion mechanism B are respectively used to extend to the specified position; the telescopic lighting and imaging system A and / or the telescopic lighting and imaging system B are respectively used to expose and shoot; the telescopic motion mechanism A and / or the telescopic motion mechanism B are also respectively used to retract after image acquisition is completed, so that the inspected seat can be transferred to the manual re-inspection station.

[0017] In the above technical solution, the seat posture error prevention mechanism adopts an L-shaped aluminum profile bracket, and the mounting height is higher than the seat cushion surface of the detected seat.

[0018] The above-mentioned deep learning-based automobile seat surface defect online visual inspection system is applicable to the deep learning-based automobile seat surface defect detection method, which includes the following steps:

[0019] Step 1), collecting and preprocessing the image of the seat surface defects to be inspected;

[0020] Step 2), using the preprocessed images to train a defect detection model;

[0021] Step 3), input the image to be detected into the trained defect detection model for defect recognition.

[0022] In the above technical solution, the preprocessing algorithms used in step 1) include: distortion correction, template matching, edge detection, adaptive threshold segmentation, binarization and Blob analysis.

[0023] In the above technical solution, step 2) is specifically as follows:

[0024] a) The seat defect features are divided into two categories: high contrast small area and low contrast large area, and a multi-layer convolutional neural network A for detecting thread ends and damage and a multi-layer convolutional neural network B for detecting oil stains and wrinkles are constructed respectively;

[0025] b) Collect defect image samples and manually annotate them, using outline strokes to mark the defect area, size, and type. The defect types are marked as thread ends, damage, oil stains, and wrinkles;

[0026] c) Importing thread end and breakage defect samples into the multi-layer convolutional neural network A, and optimizing and training the multi-layer convolutional neural network A using an optimization algorithm in combination with manual labeling information, thereby obtaining a thread end and breakage defect detection model A based on deep learning;

[0027] d) Importing the oil stain and wrinkle defect samples into the multi-layer convolutional neural network B, and optimizing and training the multi-layer convolutional neural network B using the optimization algorithm in combination with the artificial labeling information, thereby obtaining the oil stain and wrinkle defect detection model B based on deep learning;

[0028] e) Defect detection model A and defect detection model B are combined to obtain a car seat surface defect detection model.

[0029] In the above technical solution, step 3) is specifically as follows:

[0030] a) The preprocessed image to be inspected is segmented according to the multi-target detection method to generate multiple sub-images with a resolution of p×p, which are imported into the defect detection model A to determine whether there are thread ends and damage defects in the input image. If there are thread ends and damage defects, the location and size of the thread ends and damage are calculated;

[0031] b) Segment the preprocessed image to be detected according to the multi-target detection method to generate multiple sub-images with a resolution of q×q, and import them into the defect detection model B to determine whether there are oil stains and wrinkle defects in the input image. If there are oil stains and wrinkle defects, calculate the location and size of the oil stains and wrinkles;

[0032] c) Summarize the data information of thread ends, breakage, oil stains, and wrinkles, and output the defect detection results of the car seat surface.

[0033] The present invention has the following beneficial effects:

[0034] The deep learning-based online visual inspection system for automobile seat surface defects of the present invention solves the problem that traditional automobile seat inspection systems cannot cover the entire seat surface under a vertical shooting angle of view through the hardware joint design of the optical imaging system and the electronic control system, and realizes the optimal vertical full-coverage inspection field of view path planning based on different seat morphologies.

[0035] The deep learning-based online visual inspection system for automobile seat surface defects of the present invention improves the lighting uniformity and the contrast between the seat surface defect target and the skin background by optimizing the lighting source band, lighting angle design and 4-zone light source design, thereby improving the inspection effect.

[0036] The online visual detection method for automobile seat surface defects based on deep learning of the present invention adopts a robot shooting path planning technology based on the 3D morphology of the seat, optimizes the optimal robot shooting motion path, imaging distance and angle according to the surface morphology characteristics of the seat, and improves the detection rhythm.

[0037] The online visual detection method for automobile seat surface defects based on deep learning of the present invention constructs a deep learning model based on convolutional neural network, and after constructing two defect detection models respectively according to the contrast and area characteristics of seat defects, a large number of defect samples are used to continuously strengthen the training of the defect detection model, thereby effectively improving the detection capability of the defect detection model, and efficiently detecting defects with irregular defect positions and features, thereby making up for the shortcomings of traditional visual detection algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0039] Figure 1 Schematic diagram of the detection system layout.

[0040] Figure 2 Schematic diagram of the detection system workflow.

[0041] Figure 3 Schematic diagram of the imaging system's detection field of view.

[0042] Figure 4 Schematic diagram of the design principle for the angle and divergence angle of the light source.

[0043] Figure 5 Take a picture of the path planning diagram for robot B.

[0044] Figure 6 Schematic diagram of the multi-layer convolutional neural network structure used to identify thread ends and breakage defects.

[0045] Figure 7 This is a schematic diagram of the changing trend of recognition accuracy during the training of the thread end and breakage defect detection model.

[0046] Figure 8 Schematic diagram of defect detection results after defect images are input into the trained model respectively.

[0047] The reference numerals in the figures indicate:

[0048] 1-Robot A, 2-Robot lighting and imaging system A, 3-Telescopic motion mechanism A,

[0049] 4-Robot lighting and imaging system B, 5-Electronic control and image processing system, 6-Telescopic motion mechanism B, 7-Telescopic lighting and imaging system A, 8-Telescopic lighting and imaging system B,

[0050] 9-seat posture error prevention mechanism, 10-detected seat, 11-robot B, 12-display. DETAILED DESCRIPTION

[0051] The inventive concept of the present invention is:

[0052] The present invention proposes an online visual inspection system for automobile seat surface defects based on deep learning, including: two sets of robots, two sets of telescopic motion mechanisms, a lighting system, an imaging system, a seat posture error prevention mechanism, an electric control system, an image processing system and a display. Among them, the imaging system and the lighting system are each equipped with 8 sets, namely: the front robot is equipped with 1 set, the rear robot is equipped with 1 set, the left electric cylinder is equipped with 3 sets, and the right electric cylinder is equipped with 3 sets.

[0053] The in-depth learning-based online visual inspection system for automobile seat surface defects of the present invention adopts a combination scheme of a robot and a telescopic motion mechanism equipped with a camera for motion photography in terms of hardware architecture design, wherein a robot is installed before and after the seat inspection station, and the robot carries a camera along a preset path to photograph the front, back, and top of the seat from multiple angles and multiple positions. A telescopic motion mechanism is installed on each side of the seat inspection station, each telescopic motion mechanism is connected to a detection slide rail, and three cameras are installed on each detection slide rail. When the seat enters the measurement station, the seat posture error prevention mechanism will screen the seat posture to prevent the seat with incorrect angle from entering the inspection station and interfering with the robot and the telescopic motion mechanism. When the seat enters the measurement station, the telescopic motion mechanism extends to the specified position, and six cameras are exposed to take pictures. After the image acquisition is completed, the telescopic rod is retracted, and the seat continues to flow to the manual re-inspection station. This scheme can solve the on-site layout problems such as insufficient robot arm span and narrow distance between two seat stations. At the same time, the telescopic rod can be used to realize the front vertical shooting of the seat surface, and the detection effect is better than that of the squint shooting. It makes up for the blind spot problem of visual inspection in fixed camera array and single robotic arm inspection methods, and realizes online coverage inspection of the entire surface of the seat.

[0054] The deep learning-based online visual inspection system for automobile seat surface defects of the present invention analyzes the reflective spectral characteristics of defects such as the skin surface and thread ends, finds the wavelength with the largest difference in reflectivity between the skin and the defect, and uses this wavelength as the reference wavelength of the lighting source to improve the contrast between the defect target and the background.

[0055] The deep learning-based online visual inspection system for automobile seat surface defects of the present invention adopts a multi-partitioned annular light source design and multi-quadrant time-sharing triggered exposure. It can achieve multi-angle lighting at a single fixed photographing point, increase the diversity of lighting angles, and make up for the possible missed defects caused by single-angle lighting.

[0056] The deep learning-based online visual inspection system for automobile seat surface defects of the present invention is applicable to a robot shooting path planning technology based on the 3D morphology of the seat. According to the surface morphology characteristics of the seat, the optimal robot shooting motion path, imaging distance and angle are optimized, thereby improving the inspection rhythm while taking into account the inspection accuracy.

[0057] The online visual inspection system for automobile seat surface defects based on deep learning of the present invention adopts an automobile seat surface defect inspection method based on deep learning, and the inspection process is as follows:

[0058] 1) Collect the surface defect images of the inspected seat and perform preprocessing.

[0059] The preprocessing algorithms used include: distortion correction, template matching, edge detection, adaptive threshold segmentation, binarization, and Blob analysis.

[0060] 2) Use the preprocessed images to train the defect detection model.

[0061] Follow these steps:

[0062] a) According to the industry inspection standard for automotive seat surface defects, thread ends and damage defects are characterized by high contrast and small area, while oil stains and wrinkles are characterized by low contrast and large area. Therefore, the seat defect features can be divided into two categories: high contrast and small area and low contrast and large area. A multi-layer convolutional neural network A for detecting thread ends and damage and a multi-layer convolutional neural network B for detecting oil stains and wrinkles are constructed respectively;

[0063] b) Collect defect image samples and manually annotate them, using contour tracing to mark the defect area, size and type. The defect types are marked as common defects on the surface of automobile seats, such as thread ends, damage, oil stains, and wrinkles.

[0064] c) Importing thread end and breakage defect samples into the multi-layer convolutional neural network A, and optimizing and training the multi-layer convolutional neural network A using an optimization algorithm in combination with manual labeling information, thereby obtaining a thread end and breakage defect detection model A based on deep learning;

[0065] d) Importing the oil stain and wrinkle defect samples into the multi-layer convolutional neural network B, and optimizing and training the multi-layer convolutional neural network B using the optimization algorithm in combination with the artificial labeling information, thereby obtaining the oil stain and wrinkle defect detection model B based on deep learning;

[0066] e) Defect detection model A and defect detection model B are combined to obtain a car seat surface defect detection model.

[0067] 3) Input the image to be detected into the trained defect detection model for defect identification.

[0068] After preprocessing the image to be inspected, it is imported into the defect detection model for intelligent detection of surface defects of automobile seats. The main process includes:

[0069] a) The preprocessed image to be inspected is segmented according to the multi-target detection method to generate multiple sub-images with a resolution of p×p, which are imported into the defect detection model A to determine whether there are thread ends and damage defects in the input image. If there are thread ends and damage defects, the location and size of the thread ends and damage are calculated;

[0070] b) Segment the preprocessed image to be detected according to the multi-target detection method to generate multiple sub-images with a resolution of q×q, and import them into the defect detection model B to determine whether there are oil stains and wrinkle defects in the input image. If there are oil stains and wrinkle defects, calculate the location and size of the oil stains and wrinkles;

[0071] c) Summarize the data information of thread ends, breakage, oil stains, and wrinkles, and output the defect detection results of the car seat surface.

[0072] The following is a further detailed description of the online visual inspection system for surface defects of automobile seats based on deep learning of the present invention in conjunction with the accompanying drawings.

[0073] The online visual inspection system for automobile seat surface defects based on deep learning of the present invention is as follows: Figure 1 As shown, the overall architecture includes:

[0074] Robot A1, robot lighting and imaging system A2, telescopic motion mechanism A3, telescopic lighting and imaging system A7;

[0075] Robot B11, robot lighting and imaging system B4, telescopic motion mechanism B6, telescopic lighting and imaging system B8; and

[0076] Seat posture error prevention mechanism 9 , electronic control and image processing system 5 , and display 12 .

[0077] The display 12 is used to display the detection process and results; the robot A1 and the robot B11 are respectively connected to the robot lighting and imaging system A2 and the robot lighting and imaging system B4; the robot lighting and imaging system A2 and / or the robot lighting and imaging system B4 are respectively used to photograph the front, back and top of the detected seat 10 along a preset path;

[0078] A telescopic motion mechanism A3 and a telescopic motion mechanism B6 are respectively provided on both sides of the seat inspection station, and the telescopic motion mechanism A3 and the telescopic motion mechanism B6 are respectively connected to the inspection slide rails, and a telescopic lighting and imaging system A7 and a telescopic lighting and imaging system B8 are respectively provided on each inspection slide rail;

[0079] When the seat 10 to be tested enters the measuring station, the seat posture error prevention mechanism 9 is used to screen the posture of the seat 10 to be tested;

[0080] When the inspected seat 10 enters the measuring station, the telescopic motion mechanism A3 and / or the telescopic motion mechanism B6 are respectively used to extend to the specified position; the telescopic lighting and imaging system A7 and / or the telescopic lighting and imaging system B8 are respectively used to expose and shoot; the telescopic motion mechanism A3 and / or the telescopic motion mechanism B6 are also respectively used to retract after the image acquisition is completed, so that the inspected seat 10 can be transferred to the manual re-inspection station.

[0081] The workflow of the detection system is as follows Figure 2 As shown, after the seat is transferred to the inspection station on the production line, the photoelectric switch is triggered. After waiting for the manual code scanning to be completed, the PLC in the electronic control system of the electronic control and image processing system 5 sends a start detection signal to the robot and the telescopic motion mechanism. The robot moves according to the preset points. After the movement is in place, the light source is triggered to light up. The light source then triggers the camera to take pictures and uses the deep learning detection method in the image processing system to perform defect detection. After the telescopic mechanism moves to the specified position, it sends a feedback signal to the PLC. After receiving the signal, the PLC triggers the light source, and the light source triggers the camera to take pictures. After taking pictures, the deep learning detection method in the image processing system is used to perform defect detection. When the robot and the telescopic motion mechanism have completed taking pictures of all the preset points and returned to the origin, and after the algorithm completes the detection, the detection system sends a release signal to the production line through the PLC, and the detection results are displayed on the display 12.

[0082] The flow direction of the seat production line is defined as left and right. Robot A1 and robot B11 are installed at the rear and front of the seat inspection station respectively. They are responsible for carrying robot lighting and imaging system A2 and robot lighting and imaging system B4 to shoot the front, back and top of the inspected seat 10 from multiple angles and positions along the preset path. The front robot B11 moves a total of 11 points in sequence, and the rear robot A1 moves a total of 6 points in sequence. A telescopic motion mechanism (telescopic motion mechanism A3 and telescopic motion mechanism B6) is installed on each of the left and right sides of the seat inspection station. Each telescopic motion mechanism is connected to the inspection slide rail. Three cameras and light sources are installed on each inspection slide rail. When the inspected seat 10 enters the measuring station, the telescopic motion mechanism extends to the specified position, and six cameras are exposed to shoot. After the image acquisition is completed, the telescopic rod is retracted. The travel design of the telescopic motion mechanism should meet the following requirements: when fully extended, the camera field of view it carries can completely cover the side of the inspected seat 10, and when fully retracted, it will not collide with the seats circulating on the production line. The shooting field of view of each imaging system is as follows: Figure 3 shown.

[0083] The seat posture error prevention mechanism 9 adopts an L-shaped aluminum profile bracket design, and the installation height is higher than the cushion surface of the detected seat 10. It is mainly used to prevent a seat with an incorrect angle from entering the detection station and interfering with the robot and the telescopic motion mechanism.

[0084] The lighting system, including the robot lighting and imaging system A2, the robot lighting and imaging system B4, the telescopic lighting and imaging system A7, and the telescopic lighting and imaging system B8, adopts a 4-partitioned central hole annular light source. The diameter of the central hole of the annular light source is larger than the maximum aperture of the lens. The lens is fixedly connected to the light source after passing through the central hole. The 4-partitioned light source divides the annular light source into four quadrants. Each quadrant is individually triggered for exposure by the light source controller in a time-sharing manner. Under the same viewing angle, detection images with lighting in four different directions can be obtained, which increases the probability of defective targets being detected.

[0085] The design of the wavelength of the lighting source adopts the principle of maximum contrast index, finds the wavelength with the largest difference in reflectivity between the skin and the defect, and uses this wavelength as the reference wavelength of the lighting source to improve the contrast between the defect target and the background. The calculation formula of the contrast index is as follows:

[0086]

[0087] Among them, c is the contrast index, ρ1(λ), ​​ρ2(λ), ρ3(λ), ρ4(λ) are the spectral reflectance of sewing thread, oil stain, inner fabric, and seat surface skin respectively, which can be measured by spectrophotometer. k1, k2, k3 are defect weight coefficients, which are set to 1 by default. The corresponding weight coefficients can be increased according to the importance of the defects to be detected.

[0088] In order to achieve uniform illumination of the light source on the inspected plane, it is necessary to optimize the angle and divergence angle of the light source, such as Figure 4 As shown, the calculation formula is as follows:

[0089]

[0090] Among them, θ is the angle between the illumination center of the light source and the surface normal of the illuminated surface, a is 1 / 2 the divergence angle of the light source, w is the width range of the illuminated surface, WD is the working distance of the imaging system, and d is the radius of the annular light source.

[0091] The imaging system includes the robot lighting and imaging system A2, the robot lighting and imaging system B4, the telescopic lighting and imaging system A7, and the telescopic lighting and imaging system B8, which are composed of an area array industrial camera, a lens, and an image acquisition card. The area array industrial camera is used to collect detection images, which are transmitted to the image processing system through the image acquisition card for defect detection. The electronic control system is responsible for controlling the point motion control of the robot, the telescopic motion control of the telescopic motion mechanism, the light source exposure, and the camera triggering to take pictures.

[0092] The selection of imaging system parameters and the path planning of the robot need to take into account both the defect detection accuracy and the detection rhythm. Taking the camera carried by robot B11 as an example, the camera resolution used in this embodiment is 5472*3648, the shooting distance is 400mm, the shooting field of view is 437mm*291mm, and the pixel resolution is 0.08mm. If the target contour is to be stably extracted, the target needs to occupy 12 pixels. Therefore, the minimum defect that the imaging system can detect is about 1mm. The shooting path planning of robot B11 is as follows: Figure 5 As shown, robot B11 photographs 11 points along the shortest path, and the detection cycle is about 31 seconds.

[0093] The image processing system includes an electronic control and image processing system 5, which adopts a surface defect detection method for automobile seats based on deep learning. The defect detection model is a convolutional neural network based on deep learning, including: a feature extraction network, a classifier and a regressor. Each module is cascaded in sequence. The feature extraction network is used to extract features of the preprocessed image input into the model, so as to obtain the corresponding feature image. The classifier and the regressor classify and regress the feature image to obtain specific information of the defect to be detected.

[0094] The detection process is:

[0095] 1) Collecting and preprocessing the surface defect image of the seat 10 to be inspected. The preprocessing process includes:

[0096] When the inspected seat 10 enters the inspection station, the image of the inspected seat 10 is firstly subjected to distortion correction;

[0097] The rotation and translation of the seat image caused by the posture change of the detected seat 10 are compensated by template matching;

[0098] The interference of background environment such as lighting and workers walking is shielded by masking to accurately extract the ROI area;

[0099] After Gaussian filtering of the ROI area, the gradient image adaptive threshold segmentation method and canny edge detection method are used to extract the defect image, and the complete target boundary is obtained through corrosion and expansion;

[0100] 2) Use the preprocessed images to train the detection model. Here, we take the detection model training of thread ends and breakage defects as an example to explain in detail. The multi-layer convolutional neural network structure used to identify thread ends and breakage defects is as follows: Figure 6As shown in the figure, the neural network structure consists of an input layer, convolution layer 1, pooling layer 1, convolution layer 2, pooling layer 2, a fully connected layer, and a softmax classification layer. The input layer size is 128×128, the convolution kernel size of convolution layer 1 is 5×5 and the depth is 4, the convolution kernel size of convolution layer 2 is 5×5 and the depth is 8, and the fully connected layer has 2048 neural nodes. The linear correction function Leaky ReLU is used as the activation function:

[0101]

[0102] Among them, a i The meaning of is a constant coefficient, usually set to 0.01 or other values ​​close to zero. i is the function's independent variable.

[0103] During the training process, the time decay strategy is used to adjust the learning rate. The learning rate gradually decreases with the increase of the number of training iterations. The formula is expressed as:

[0104] α t =α0×(1+γ×t) -p (5)

[0105] In the formula, α t is the learning rate of the tth iteration, α0 is the initial learning rate, γ is the decay rate, and p is a positive real number used to control the decay speed.

[0106] 1000 thread ends and breakage defect samples were used as one round of training data, and a total of 400 iterations of training were performed. During the training process, the recognition accuracy of the thread end and breakage defect detection model changed with the number of training iterations. Figure 7 As shown in the figure, the defect recognition accuracy continues to improve with the increase in the number of iterations. After 400 iterations of training, the recognition accuracy has exceeded 95%. At this time, the thread head and damage defect detection model has been preliminarily trained, and the number of samples can be increased for iterative training in the later stage. Similarly, the oil stain and wrinkle detection model can also be trained in this way.

[0107] 3) Input the image to be detected into the trained defect detection model for defect identification.

[0108] like Figure 8 As shown in the figure, the thread end and oil stain defect images are used as input images, and the trained thread end and damage defect detection models and the oil stain and wrinkle defect detection models are used to perform defect recognition respectively, and the position, area, confidence and other data of defect detection can be obtained. The areas of thread ends and damages are 945 pixels and 2843 pixels respectively.

[0109] The deep learning-based online visual inspection system for automobile seat surface defects of the present invention solves the problem that traditional automobile seat inspection systems cannot cover the entire seat surface under a vertical shooting angle of view through the hardware joint design of the optical imaging system and the electronic control system, and realizes the optimal vertical full-coverage inspection field of view path planning based on different seat morphologies.

[0110] The deep learning-based online visual inspection system for automobile seat surface defects of the present invention improves the lighting uniformity and the contrast between the seat surface defect target and the skin background by optimizing the lighting source band, lighting angle design and 4-zone light source design, thereby improving the inspection effect.

[0111] The online visual detection method for automobile seat surface defects based on deep learning of the present invention adopts a robot shooting path planning technology based on the 3D morphology of the seat, optimizes the optimal robot shooting motion path, imaging distance and angle according to the surface morphology characteristics of the seat, and improves the detection rhythm.

[0112] The online visual detection method for automobile seat surface defects based on deep learning of the present invention constructs a deep learning model based on convolutional neural network, and after constructing two defect detection models respectively according to the contrast and area characteristics of seat defects, a large number of defect samples are used to continuously strengthen the training of the defect detection model, thereby effectively improving the detection capability of the defect detection model, and efficiently detecting defects with irregular defect positions and features, thereby making up for the shortcomings of traditional visual detection algorithms.

[0113] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. An online visual inspection system for automobile seat surface defects based on deep learning, characterized in that: include: Robot A (1), robot lighting and imaging system A (2), telescopic motion mechanism A (3), telescopic lighting and imaging system A (7); A robot B (11), a robot lighting and imaging system B (4), a telescopic motion mechanism B (6), a telescopic lighting and imaging system B (8); and A seat posture error prevention mechanism (9), an electric control and image processing system (5) and a display (12); the display (12) is used to display the detection process and results; the electric control and image processing system (5) is used to send a detection start signal to control the robot A (1) and / or the robot B (11) to move according to a preset point position; A robot A (1) and a robot B (11) are respectively arranged before and after the seat inspection station, and the robot A (1) and the robot B (11) are respectively connected to a robot lighting and imaging system A (2) and a robot lighting and imaging system B (4); the robot lighting and imaging system A (2) and / or the robot lighting and imaging system B (4) are respectively used to photograph the front, back and top of the inspected seat (10) along a preset path; A telescopic motion mechanism A (3) and a telescopic motion mechanism B (6) are respectively provided on both sides of the seat inspection station, the telescopic motion mechanism A (3) and the telescopic motion mechanism B (6) are respectively connected to the inspection slide rails, and a telescopic lighting and imaging system A (7) and a telescopic lighting and imaging system B (8) are respectively provided on each inspection slide rail; When the seat (10) to be tested enters the measuring station, the seat posture error prevention mechanism (9) is used to screen the posture of the seat (10) to be tested; When the seat (10) to be inspected enters the measuring station, the telescopic movement mechanism A (3) and / or the telescopic movement mechanism B (6) are respectively used to extend to a specified position; the telescopic lighting and imaging system A (7) and / or the telescopic lighting and imaging system B (8) are respectively used to expose and photograph; the telescopic movement mechanism A (3) and / or the telescopic movement mechanism B (6) are also respectively used to retract after image acquisition is completed, so that the seat (10) to be inspected can be transferred to the manual re-inspection station.

2. The online visual inspection system for automobile seat surface defects based on deep learning according to claim 1 is characterized in that: The seat posture error prevention mechanism (9) adopts an L-shaped aluminum profile bracket, and the mounting height is higher than the cushion surface of the detected seat (10).

3. The deep learning-based automobile seat surface defect detection method applicable to the deep learning-based automobile seat surface defect online visual detection system according to claim 1 or 2 is characterized in that: The following steps are involved: Step 1), collecting and preprocessing the surface defect image of the seat (10) to be inspected; Step 2), using the preprocessed images to train a defect detection model; Step 3), input the image to be detected into the trained defect detection model for defect recognition.

4. The method for detecting surface defects of automobile seats based on deep learning according to claim 3 is characterized in that: The preprocessing algorithms used in step 1) include: distortion correction, template matching, edge detection, adaptive threshold segmentation, binarization and Blob analysis.

5. The method for detecting surface defects of automobile seats based on deep learning according to claim 3 is characterized in that: Step 2) is specifically as follows: a) The seat defect features are divided into two categories: high contrast small area and low contrast large area, and a multi-layer convolutional neural network A for detecting thread ends and damage and a multi-layer convolutional neural network B for detecting oil stains and wrinkles are constructed respectively; b) Collect defect image samples and manually annotate them, using outline strokes to mark the defect area, size, and type. The defect types are marked as thread ends, damage, oil stains, and wrinkles; c) Importing thread end and breakage defect samples into the multi-layer convolutional neural network A, and optimizing and training the multi-layer convolutional neural network A using an optimization algorithm in combination with manual labeling information, thereby obtaining a thread end and breakage defect detection model A based on deep learning; d) Importing the oil stain and wrinkle defect samples into the multi-layer convolutional neural network B, and optimizing and training the multi-layer convolutional neural network B using the optimization algorithm in combination with the artificial labeling information, thereby obtaining the oil stain and wrinkle defect detection model B based on deep learning; e) Defect detection model A and defect detection model B are combined to obtain a car seat surface defect detection model.

6. The method for detecting surface defects of automobile seats based on deep learning according to claim 5, characterized in that: Step 3) is specifically: a) The preprocessed image to be inspected is segmented according to the multi-target detection method to generate multiple sub-images with a resolution of p×p, which are imported into the defect detection model A to determine whether there are thread ends and damage defects in the input image. If there are thread ends and damage defects, the location and size of the thread ends and damage are calculated; b) Segment the preprocessed image to be detected according to the multi-target detection method to generate multiple sub-images with a resolution of q×q, and import them into the defect detection model B to determine whether there are oil stains and wrinkle defects in the input image. If there are oil stains and wrinkle defects, calculate the location and size of the oil stains and wrinkles; c) Summarize the data information of thread ends, breakage, oil stains, and wrinkles, and output the defect detection results of the car seat surface.

Citation Information

Patent Citations

  • Automobile seat surface defect detection system based on machine vision as well as detection method

    CN105806849A

  • Seat surface defect detecting system and method based on machine vision

    CN106226325A

  • A Defect Detection Method for Car Seats Based on Multi-Feature Fusion Machine Learning

    CN112232399B

Cited By

  • Automatic automobile framework production defect detection system based on visual identification

    CN120411098A

  • Automatic detection system for automobile frame production defects based on visual recognition

    CN120411098B

  • Automobile seat quality multi-dimensional detection and identification method based on AI vision

    CN120635042A

  • An AI vision-based multi-dimensional detection and identification method for automobile seat quality

    CN120635042B