Surface defect detection model training method and surface defect detection method and system
Through the deep learning-based surface defect detection model training method, combined with image capture and deep learning computing modules, automatic multi-angle detection is realized, solving the problems of low detection efficiency and poor accuracy in the existing technology, and significantly improving the detection efficiency and accuracy.
Patent Information
- Application Number
- CN202311306293.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-10
- Publication Date
- 2025-04-11
AI Technical Summary
In the prior art, surface defect detection relies on manpower or robotic arms, is inefficient and prone to missed inspection, and the automatic detection production line does not control the light source, which affects the detection accuracy.
A surface defect detection model training method based on deep learning is proposed. By obtaining the training sample image set, semantic segmentation and normalization processing, a model that can automatically detect surface defects is trained, and combined with the image capture module and the deep learning operation module, multi-angle detection is realized, and training data is expanded by automatically obtaining web page data.
It realizes automatic and multi-angle detection of surface defects, significantly improves detection efficiency and accuracy, reduces human errors, and is suitable for different products and working conditions.
Smart Images
Figure CN119810583A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of surface defect detection, and in particular to a surface defect detection model training method and a surface defect detection method and system. Background Art
[0002] At present, surface inspection mainly relies on manual work or the assistance of robotic arms, which takes a long time and is inefficient. For the manufacturing industry, medical equipment industry, manufacturing industry, printing industry, packaging industry, and food industry, product quality has an important impact on customer impression and product function. Quality control is a key process to ensure that qualified and defect-free products are shipped. However, the traditional quality control process is usually visually inspected by workers, who need to hold the product and check for defects. This dull and repetitive process requires a high degree of reliance on the worker's experience and concentration, and is more likely to lead to missed inspections due to human errors. When faced with a large amount of inspection work, the efficiency of this traditional inspection method is significantly low.
[0003] For example, in the construction industry, visual inspection of welds is mainly carried out manually, which has low inspection efficiency and is prone to missing welding defects such as pores, welding slag, surface cracks, etc. that are not easy to see. As a result, welding defects in building structural parts will expand over time, easily increasing the risk of collapse.
[0004] Currently, many automatic inspection production lines have poor control over light sources and use fixed light brightness to cope with different product types. Surface inspection has very high requirements for light sources. Too low brightness or overexposure will make surface contour features difficult to be analyzed by computers. Summary of the invention
[0005] The purpose of the present invention is to overcome at least one defect in the prior art and to propose a surface defect detection model training method and a surface defect detection method and system. Through the solution of the present invention, surface defects can be detected automatically and from multiple angles and the detection efficiency and accuracy can be greatly improved.
[0006] To achieve the above object, the present invention provides a surface defect detection model training method, the method comprising:
[0007] Obtain a training sample image set;
[0008] Input the training sample images into the deep learning-based surface defect detection model;
[0009] The surface defect detection model based on deep learning performs semantic segmentation on the input training sample image to obtain the original feature map, and normalizes the original feature map to obtain the normalized feature map;
[0010] The surface defect detection model based on deep learning is trained using the normalized feature map to obtain a trained surface defect detection model based on deep learning.
[0011] In some embodiments, before the step of performing semantic segmentation on the surface image of the training sample image to obtain the original feature map, the following step is also included: performing denoising on the surface image of the training sample image.
[0012] In some embodiments, semantic segmentation is performed on the denoised surface image to obtain an original feature map.
[0013] In some embodiments, the formula for normalizing the original feature map is: P(pixel) = (D(pixel)-D(min)) / (D(max)-D(min)), where: P(pixel) is the degree of defect of the pixel, D(pixel) is the measurement value of the pixel on the preset feature, D(min) is the minimum measurement value of all pixels in the entire image on the preset feature, and D(max) is the maximum measurement value of all pixels in the entire image on the preset feature.
[0014] In some embodiments, the training sample image set includes a first training sample image set and a second training sample image set, wherein the first training sample image set includes several images without surface defects, and the second training sample image set includes several images with surface defects; the images with surface defects include marked surface defect images and / or unmarked surface defect images.
[0015] In some embodiments, the training sample image set is obtained by photographing with an image capture module;
[0016] and / or,
[0017] The training sample image set is obtained by automatically acquiring web page data.
[0018] In some embodiments, the training sample image set is obtained by automatically acquiring web page data, and the steps include:
[0019] Obtain HTML content: Send HTTP request to the server of the target website to obtain HTML content of scientific research web pages or other web pages related to surface defects;
[0020] Parsing HTML: After obtaining the HTML content, parse it to extract the required information;
[0021] Select and extract data: Use selectors to locate specific HTML elements that contain the required data, and then extract image database information from these elements.
[0022] The steps of obtaining a training sample image set by automatically acquiring web page data include:
[0023] Obtain HTML content: Send HTTP request to the server of the target website to obtain HTML content of scientific research web pages or other web pages related to surface defects;
[0024] Parsing HTML: After obtaining the HTML content, parse it to extract the required information;
[0025] Select and extract data: Use selectors to locate specific HTML elements that contain the required data, and then extract image database information from these elements.
[0026] The present invention also discloses a surface defect detection method, comprising the following steps:
[0027] Acquire a surface image of the object to be measured;
[0028] The surface image of the object to be tested is input into a surface defect detection model based on deep learning obtained by training using the surface defect detection model training method as described above, and the surface defect detection model based on deep learning outputs a defect detection result.
[0029] In some embodiments, the defect detection result includes a normal or defect result, and when the defect detection result is a defect result, the defect detection result also includes: at least one of: defect type, defect location, defect area, and defect number.
[0030] In some embodiments, the surface image of the object to be measured is obtained by an image capture module, the image capture module includes a camera with an outer ring equipped with a ring light source, and before the surface image of the object to be measured is obtained by the image capture module, the method further includes: optimizing the parameters of the image capture module;
[0031] Optimizing the parameters of the image capture module specifically includes: receiving a surface image of the object to be measured taken by the image capture module, and obtaining the current brightness of each light source and the parameters of the camera, analyzing the received surface image, and determining whether the clarity of all contours of the object to be measured meets the requirements; if not, feedback is provided to adjust the brightness of each light source of the ring light source and the parameters of the camera until the surface image of the object to be measured taken by the image capture module meets the requirements, wherein the parameters of the camera include at least one of shooting frequency, focal length, exposure, white balance, contrast, and saturation.
[0032] The present invention also discloses a surface defect detection system, comprising:
[0033] An image capturing module, wherein the image capturing module is used to capture all surfaces of the object to be measured to obtain a surface image of the object to be measured;
[0034] A deep learning operation module, wherein the deep learning operation module is provided with a surface defect detection model based on deep learning obtained by training using the surface defect detection model training method as described above, wherein the surface defect detection model based on deep learning is used to receive a surface image of an object to be tested and output a defect detection result.
[0035] In some embodiments, the surface defect detection system of the present invention further includes a monitoring platform, which is used to display the defect detection results output by the deep learning operation module;
[0036] The defect detection result includes a normal result or a defect result, and when the defect detection result is a defect result, the defect detection result also includes: at least one of: defect type, defect position, defect area, and defect number;
[0037] It also includes an intelligent data module, which is used to obtain and store the training sample image set taken by the image capture module, and / or the intelligent data module is used to obtain and store the training sample image set by automatically acquiring web page data.
[0038] In some embodiments, the image capture module includes a camera with an outer ring equipped with a ring light source, the ring light source includes a number of light-emitting bodies distributed around the camera, each light-emitting body is connected to a deep learning operation module, the brightness of each light-emitting body is independently controlled by the deep learning operation module, the deep learning operation module is used to receive the surface image of the object to be measured taken by the image capture module, and obtain the current brightness of each light-emitting body and the parameters of the camera. The deep learning operation module is also used to analyze the received surface image to determine whether the clarity of all contours of the object to be measured meets the requirements. If not, the brightness of each light-emitting body of the ring light source and the parameters of the camera are adjusted by feedback until the surface image of the object to be measured taken by the image capture module meets the requirements. The parameters of the camera include at least one of shooting frequency, focal length, exposure, white balance, contrast, and saturation.
[0039] The present invention has at least the following beneficial effects:
[0040] The present invention uses one or more image capture modules. The image capture modules in the detection area shoot all surfaces of the detection object. The deep learning algorithm compares the defect data pre-collected in the intelligent data module to calculate the probability of defects in each pixel. When the defects of the detection object are detected, the monitoring platform will display the location and type of the surface defects. The present invention can automatically and multi-angle detect surface defects and greatly improve the detection efficiency and accuracy.
[0041] Furthermore, the intelligent deep learning automatic detection method of the present invention can be applied to different situations, can add training data of different surfaces, and can be suitable for fixtures designed for different working conditions to perform multi-angle detection of different products.
[0042] The present invention proposes a light source specific position brightness adaptive technology, which can change the brightness of the specific light source position in response to different products or surfaces, so that the brightness of the detected surface is consistent, greatly improving the detection accuracy.
[0043] The present invention proposes a technology that automatically obtains web page data to obtain surface defect image data and non-surface defect image data and stores them in an intelligent database. By automatically obtaining a large number of diversified real network pictures, the training data of the intelligent database can be expanded to improve the detection accuracy. It can also initially obtain pictures of multiple products or surfaces for training purposes, thereby improving the wide application of the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0045] Figure 1 A flow chart of a surface defect detection model training method provided by an embodiment of the present invention;
[0046] Figure 2 A flow chart of a surface defect detection method provided by an embodiment of the present invention;
[0047] Figure 3 A principle block diagram of a surface defect detection system provided by an embodiment of the present invention;
[0048] Figure 4 A schematic diagram of an intelligent deep learning automatic detection system for surface defects of injection molded parts at all angles provided by a specific embodiment of the present invention;
[0049] Figure 5 A schematic diagram of an intelligent deep learning automatic detection system for surface defects of full-angle I-beam butt welding welds provided by a specific embodiment of the present invention;
[0050] Figure 6a 6a is the normalized feature map of the sample without surface defects, and 6b is the normalized feature map of the sample with surface defects. DETAILED DESCRIPTION
[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0052] Embodiment 1
[0053] See also Figure 1 , an embodiment of the present invention provides a surface defect detection model training method, the method comprising:
[0054] Obtain a training sample image set;
[0055] Input the training sample images into the deep learning-based surface defect detection model;
[0056] The surface defect detection model based on deep learning performs semantic segmentation on the input training sample image to obtain the original feature map, and normalizes the original feature map to obtain the normalized feature map;
[0057] The surface defect detection model based on deep learning is trained using the normalized feature map to obtain a trained surface defect detection model based on deep learning.
[0058] In some embodiments, before the step of performing semantic segmentation on the surface image of the training sample image to obtain the original feature map, the following step is also included: performing denoising on the surface image of the training sample image.
[0059] In some embodiments, semantic segmentation is performed on the denoised surface image to obtain an original feature map.
[0060] In some embodiments, the surface image of the object to be tested is subjected to denoising, specifically comprising: calculating the surface image of the object to be tested with a function algorithm and determining whether each pixel has noise, and if so, filtering the noise. The function algorithm includes but is not limited to a median filter algorithm, a mean filter algorithm, a Gaussian filter algorithm, a wavelet denoising algorithm, a low-rank matrix recovery algorithm, etc., to reduce noise points and noise in the image, so as to improve the quality and clarity of the image, and output the denoised image to the next convolution layer.
[0061] After image processing, the image is obtained without surface noise, and then semantic segmentation is performed to obtain the original feature map, which can be a grayscale image.
[0062] In some embodiments, the formula for normalizing the original feature map is: P(pixel) = (D(pixel)-D(min)) / (D(max)-D(min)), where: P(pixel) is the degree of defect of the pixel, D(pixel) is a certain metric or feature of the pixel, indicating the degree of defect of the pixel, the metric value of the pixel on the preset feature can be the grayscale value, texture feature or color information of the pixel, D(min) is the metric value of the pixel with the smallest metric value in the entire image, that is, the minimum metric value of all pixels in the entire image on the preset feature, and D(max) is the metric value of the pixel with the largest metric value in the entire image, that is, the maximum metric value of all pixels in the entire image on the preset feature.
[0063] In some embodiments, the present invention first removes noise before processing the image into a grayscale image. The meaning of a grayscale image is that the black and white values of each pixel can be analyzed, such as Figure 6a and Figure 6b As shown, 6a is the normalized feature map of the sample without surface defects, and 6b is the normalized feature map of the sample with surface defects, where the closer to 1, the darker it is, and the closer to 0, the whiter it is.
[0064] The present invention uses a semantic segmentation algorithm to grayscale the surface image of the detection object captured by the image capture module, and then performs a subsequent normalization process, without requiring a large amount of learning data. During the training process of the deep learning operation module, the image capture module can be allowed to capture pictures as defect-free image learning data of the deep learning operation module. When there is a difference after comparison with the defect-free image learning data, it will be determined whether it is a defective part.
[0065] Since the image has been processed for noise, and the image brightness has been optimized through the automatic correction of the light source brightness, the uncertainty of the image noise and brightness has been removed. Therefore, the present invention can simply map the metric value of each pixel to a range between 0 and 1 to represent the relative degree of defect probability. The convolution layer of the deep learning operation module normalizes the metric value range of the entire image according to the experience of the training data, maps the minimum value to 0, the maximum value to 1, and linearly interpolates the intermediate value between 0 and 1, so that the output result is a defect image or a non-defect image.
[0066] In some embodiments, the training sample image set includes a first training sample image set and a second training sample image set, wherein the first training sample image set includes a number of images without surface defects, and the second training sample image set includes a number of images with surface defects.
[0067] In some embodiments, the image with the surface defects includes a marked surface defect image and / or an unmarked surface defect image.
[0068] The present invention supports supervised learning and unsupervised learning. In surface defect detection, users can choose to use supervised learning. Supervised learning requires labeled training data, that is, images containing correct surface defect labels. In practice, the image capture module is used to shoot different angles of the defect-free detection surface during the training process. For the defect detection object, the surface defect image is shot through the image capture module, and the defect area and type are marked. Finally, two sets of training data with and without defects are obtained and transmitted to the intelligent data module. Then, image processing, semantic segmentation operations and normalization processing are performed to output the detection results.
[0069] In surface defect detection, users can choose to use unsupervised learning. Unsupervised learning collects unlabeled surface defect image data and non-surface defect image data, which can be used to discover and analyze unknown defect patterns, and then perform image processing and semantic segmentation operations. The actual approach requires that during the training process, the defective and non-defective detection surfaces are photographed at different angles with the image capture module and transmitted to the intelligent data module. The deep learning algorithm will compare the unlabeled surface defect-free images of the training data from each pixel. Users can set the detection sensitivity digital critical value. The lower the detection sensitivity number, the more pixels are allowed to be judged as defects. The higher the detection sensitivity number, the more pixels are allowed to be judged as defects. Only when the detection sensitivity digital critical value is exceeded is it considered a defective image. Finally, the results of normal or defective parts, defect types, defect locations, defect areas, number of defects and other signals can be output to the monitoring platform.
[0070] The marked surface defect image has a defect label for marking the defect area and type therein.
[0071] In some embodiments, the training sample image set is obtained by photographing with an image capture module.
[0072] To address the problem that it is very time-consuming and inefficient to manually control an image capture module to capture the surface and defects of each product as training data for machine learning, the present invention proposes a solution: a training sample image set is obtained by automatically acquiring web page data.
[0073] The present invention can obtain surface defect image data and non-surface defect image data by automatically acquiring web page data and store them in an intelligent database to provide defect image data and non-surface defect image data of different types of surfaces for deep learning calculation module training.
[0074] The training sample image set is obtained by automatically acquiring web page data, and the steps are as follows: 1. Obtain HTML content: Use tools or programming languages such as the Requests library in Python or the Axios library in JavaScript to send HTTP requests to the server of the target website to obtain the HTML content of scientific research web pages or other web pages related to surface defects. 2. Parse HTML: After obtaining the HTML content, use an HTML parsing library (including but not limited to BeautifulSoup in Python or Cheerio in JavaScript) to parse it to extract the required data. These libraries provide surface defect image data and non-surface defect image data for navigating HTML structures, finding specific elements, and extracting the required data. 3. Select and extract data: Use selectors (including but not limited to CSS selectors or XPath expressions) to locate specific HTML elements containing the required data, and then extract image database information from these elements.
[0075] The present invention obtains surface defect image data and non-surface defect image data by automatically acquiring web page data and stores them in an intelligent database. By automatically acquiring a large number of diversified real network pictures, the training data of the intelligent database can be expanded to improve the detection accuracy. It can also preliminarily acquire pictures of multiple products or surfaces for training purposes, thereby improving the wide application of the algorithm.
[0076] Embodiment 2
[0077] See also Figure 2 The embodiment of the present invention also discloses a surface defect detection method, comprising the following steps:
[0078] Acquire a surface image of the object to be measured;
[0079] The surface image of the object to be tested is input into a surface defect detection model based on deep learning obtained by training using the surface defect detection model training method as described in Example 1, and the surface defect detection model based on deep learning outputs a defect detection result. The defect detection result includes a normal or defect result, and when the defect detection result is a defect result, the defect detection result also includes: at least one of defect type, defect location, defect area, and defect number.
[0080] In some embodiments, a surface defect detection model based on deep learning denoises an input surface image of an object to be tested, performs semantic segmentation on the denoised surface image to obtain an original feature map, normalizes the original feature map to obtain a normalized feature map, classifies the normalized feature map, and outputs a defect detection result.
[0081] In some embodiments, the surface image of the object to be measured is obtained through an image capture module, and the image capture module includes a camera with an outer ring equipped with a ring light source. Before obtaining the surface image of the object to be measured through the image capture module, it also includes: optimizing the parameters of the image capture module.
[0082] Optimizing the parameters of the image capture module specifically includes: receiving the surface image of the object to be tested taken by the image capture module, and obtaining the current brightness of each light source and the parameters of the camera, analyzing the received surface image, and determining whether the clarity of all contours of the object to be tested meets the requirements. If not, feedback is provided to adjust the brightness of each light source of the ring light source and the parameters of the camera (such as if the contour of a certain part of the image is not clear, then adjust the brightness of the light source corresponding to the part) until the surface image of the object to be tested taken by the image capture module meets the requirements, and the parameters of the camera include at least one of the shooting frequency, focal length, exposure, white balance, contrast, and saturation. For example, the actual brightness signal of the object to be tested sensed by the photosensitive element of the two-dimensional visible light camera indicates that there is an exposure problem in the shooting area. The deep learning operation module determines the actual brightness of the object to be tested. When it is determined to be exposed, the shooting exposure value of the two-dimensional visible light camera will be automatically reduced.
[0083] Embodiment 3
[0084] See also Figures 3 to 5 The embodiment of the present invention further discloses a surface defect detection system, comprising:
[0085] An image capturing module, wherein the image capturing module is used to capture all surfaces of the object to be measured to obtain a surface image of the object to be measured;
[0086] A deep learning operation module, wherein the deep learning operation module is provided with a surface defect detection model based on deep learning obtained by training using the surface defect detection model training method as described in Example 1, wherein the surface defect detection model based on deep learning is used to receive a surface image of an object to be tested and output a defect detection result.
[0087] The defect detection result includes a normal result or a defect result, and when the defect detection result is a defect result, the defect detection result also includes: at least one of: defect type, defect position, defect area, and defect number.
[0088] In some embodiments, the surface defect detection system of the present invention also includes a monitoring platform, which is used to display the defect detection results output by the deep learning operation module, such as displaying the defect detection results in real time on the monitoring platform, which may display information including but not limited to defect or no defect results, defect type, defect location, defect area, number of defects, etc.
[0089] In some embodiments, the surface defect detection system of the present invention also includes an intelligent data module, which is used to store the supervised learning and unsupervised learning training data and web page image data of the deep learning operation module, and then feed it back to the deep learning operation module for calculation.
[0090] In some embodiments, the intelligent data module is used to obtain and store the training sample image set taken by the image capture module, and / or the intelligent data module is used to obtain and store the training sample image set by automatically obtaining web page data.
[0091] In some embodiments, the image capture module includes a camera with an outer ring equipped with a ring light source, the ring light source includes a number of light-emitting bodies distributed around the camera, each light-emitting body is connected to a deep learning operation module, the brightness of each light-emitting body is independently controlled by the deep learning operation module, the deep learning operation module is used to receive the surface image of the object to be measured taken by the image capture module, and obtain the current brightness of each light-emitting body and the parameters of the camera. The deep learning operation module is also used to analyze the received surface image to determine whether the clarity of all contours of the object to be measured meets the requirements. If not, the brightness of each light-emitting body of the ring light source and the parameters of the camera are adjusted by feedback until the surface image of the object to be measured taken by the image capture module meets the requirements. The parameters of the camera include at least one of shooting frequency, focal length, exposure, white balance, contrast, and saturation.
[0092] In some embodiments, the image capture module includes a two-dimensional visible light camera to capture all surfaces of the object to be detected. The configurable parameters of the two-dimensional visible light camera include shooting frequency, focal length, exposure, white balance, contrast, and saturation. The parameters are set to the best clarity, resolution, and light and dark environment so that the deep learning operation module can calculate accurate results. The outer ring of each two-dimensional visible light camera is equipped with an annular light source, which is composed of a ring of dense light-emitting diodes and connected to the deep learning operation module. The brightness of each light-emitting diode can be independently controlled by the deep learning operation module. The actual brightness of the object to be detected will be sensed through the photosensitive element of the two-dimensional visible light camera, and the signal will be transmitted to the deep learning operation module. The deep learning operation module determines whether the actual brightness of the object to be detected can enable the image capture module to clearly capture all contours of the detection object. If the actual brightness of a part of the detection object is too low, the deep learning operation module will control the light-emitting diode closest to the part of the annular light source to increase the brightness, and vice versa, so that the two-dimensional visible light camera can capture high-resolution images. The image capture module can be applied to different scenarios, and the number of image capture modules and the design of the fixture for fixing the image capture are not limited by this patent.
[0093] The deep learning computing module can automatically collect parameters including but not limited to focal length, exposure, white balance, contrast, saturation, etc. through the photosensitive unit and autofocus unit of the two-dimensional visible light camera, automatically calculate and optimize the parameters in a closed loop, and use them as feedback to control the parameters of the two-dimensional visible light camera. All two-dimensional visible light cameras synchronously correct the parameters of the two-dimensional visible light camera.
[0094] The deep learning operation module receives the actual brightness signal of the object to be detected sensed by the photosensitive element of the two-dimensional visible light camera and parameters including but not limited to focal length, exposure, white balance, contrast, saturation, etc. The deep learning operation module determines the actual brightness of the object to be detected. If the judgment result is that the image capture module cannot clearly capture all the contours of the detection object, the deep learning operation module will control the brightness of the light-emitting diode closest to the part of the ring light source and automatically calculate and close-loop optimize the parameters including but not limited to focal length, exposure, white balance, contrast, saturation, etc., so as to feed back to control the above parameters of the two-dimensional visible light camera. All two-dimensional visible light cameras synchronously correct the parameters of the two-dimensional visible light camera so that the two-dimensional visible light camera can capture high-resolution images.
[0095] The above scheme is described in detail below through specific examples.
[0096] A specific embodiment uses an injection molded part inspection production line as an illustration of the invention content. This embodiment utilizes the full-angle surface defect automatic detection system based on intelligent deep learning of the present invention to automatically detect surface defects of injection molded parts, which can automatically detect to improve detection accuracy and efficiency.
[0097] See also Figure 4 The full-angle surface defect automatic detection system based on intelligent deep learning includes an image capture module, a deep learning calculation module, an intelligent data module, a monitoring platform and a conveying device.
[0098] A conveying device in one embodiment includes a transparent conveyor belt, on which a separating guide bar 3 for separating products and a movable guide bar 4 for sorting products are provided. The separating guide bar is originally in a stationary state, and when an injection molded part is about to pass in front of the separating guide bar, the separating guide bar will pop out to block the injection molded part. The present invention is not limited to the above-mentioned conveying device.
[0099] In this embodiment, the injection molded parts 1 to be inspected are conveyed to the inspection area at different angles through a transparent conveyor belt 2 and sorted by a movable guide strip 4, and qualified injection molded parts 5 and unqualified injection molded parts 6 are sorted out. The deep learning operation module grayscales the surface image of the inspection object photographed by the image capture module using a semantic segmentation algorithm, and performs subsequent image processing, and does not require a large amount of learning data in the process. In this embodiment, the user chooses to use unsupervised learning, which collects unlabeled surface defect image data of injection molded parts, which can be used to discover and analyze unknown defect patterns, and then perform image processing and semantic segmentation operations. During the training process, the user takes pictures of defective and non-defective injection molded parts at different angles with the image capture module and transmits them to the intelligent data module. In addition, the intelligent data module automatically obtains the injection molding defect image data and the non-surface defect image data by automatically acquiring the web page data and stores them in the intelligent database to provide injection molding defect image data and non-surface defect image data of different types of surfaces for the deep learning operation module to train. The deep learning algorithm compares each pixel with the unlabeled defect-free images of the injection molded parts surface in the training data. The user sets the detection sensitivity digital critical value. The lower the detection sensitivity number, the smaller the number of pixels are allowed to be judged as defects. The higher the detection sensitivity number, the larger the number of pixels are allowed to be judged as defects. Only when the detection sensitivity digital critical value is exceeded is it considered as a defective image, and the final detection result is output.
[0100] During the inspection process, first, the injection molded parts 1 to be inspected are sent to the conveyor belt at an irregular angle and pass through the partition guide strip 3. The partition guide strip 3 will separate the injection molded parts 1 to be inspected to prevent the surfaces of the two injection molded parts 1 to be inspected from being too close, which would cause the image capture module to fail to capture all surfaces. The injection molded parts 1 to be inspected are arranged randomly and sent to the inspection area. Cameras are set up in six directions: up, down, left, right, front, and back in the inspection area to capture the injection molded parts 1 at all angles. Figure 4 The camera set at the position marked ① is the left camera, the camera set at the position marked ② is the right camera, the cameras set at the position marked ③ are the upper camera and the lower camera, the camera set at the position marked ④ is the front camera, and the rear camera is Figure 4 The position of each camera can be adjusted as needed. Figure 4 Position shown.
[0101] The configurable parameters of the two-dimensional visible light camera include shooting frequency, focal length, exposure, white balance, contrast, and saturation. The deep learning operation module automatically calculates and optimizes the parameters in a closed loop, sets the parameters to the best clarity, resolution, and light and dark environment, and uses them as feedback to control the parameters of the two-dimensional visible light camera. All two-dimensional visible light cameras synchronously correct the parameters of the two-dimensional visible light camera. This enables the deep learning operation module to calculate accurate results. The outer ring of each two-dimensional visible light camera is equipped with a ring light source, which is composed of dense light-emitting diodes and connected to the deep learning operation module. The brightness of each light-emitting diode can be independently controlled by the deep learning operation module. When the injection molded part passes through the turning point of the conveyor belt, the actual brightness of the object to be detected will be sensed by the photosensitive element of the two-dimensional visible light camera. It is found that the surface below the injection molded part is darker, and the signal is transmitted to the deep learning operation module. The deep learning operation module determines that the actual brightness of the injection molded part is darker, so that the image capture module cannot clearly capture all the contours of the detection object. The deep learning operation module will control the light-emitting diode closest to the ring light source to increase the brightness, so that the two-dimensional visible light camera can capture high-resolution images. The image capture module can be applied to different scenarios, and the number of image capture modules and the design of the fixture for fixing the image capture are not limited by this patent. All two-dimensional visible light cameras capture all surfaces of the injection molded part. The deep learning operation module uses image processing to calculate the injection molded part image with a function algorithm and determines whether each pixel is noise, and filters the noise. The function algorithm includes but is not limited to a median filter algorithm, a mean filter algorithm, a Gaussian filter algorithm, a wavelet noise reduction algorithm, a low-rank matrix recovery algorithm, etc., which reduces the noise points and noise in the image, improves the image quality and clarity, and outputs a high-quality and clear image to the next convolution layer. After image processing, the image obtains an image without surface noise, and then semantic segmentation is performed to obtain a grayscale image, and then normalization is performed to obtain a feature map. Finally, the deep learning operation module outputs the detection result according to the feature map. For example, if there are 5 consecutive pixel defects on one surface of the injection molded part, the probability is 1, and the final output result is a defective image to the monitoring platform. The monitoring platform responds to the deep learning computing module's analysis results on the surface of the inspection object and displays them in real time on the monitoring platform, showing the defect results, the defect type is a short-shot defect, the defect location is a corner location, the defect area is 5 mm, and the number of defects is 1.
[0102] Another specific embodiment uses the detection of an I-beam 7 butt-welded weld as an illustration of the invention content. This embodiment utilizes the full-angle surface defect automatic detection system based on intelligent deep learning of the present invention to automatically detect surface defects of the I-beam butt-welded weld, which can automatically detect to improve detection accuracy and efficiency.
[0103] The full-angle surface defect automatic detection system based on intelligent deep learning includes image capture module, deep learning calculation module, intelligent data module and monitoring platform. Figure 5 In this embodiment, the weld of the I-beam butt welding is deep, and the worker needs to weld 20 layers of welds back and forth to fill the weld. While the worker is welding, the system will perform weld detection at the same time. The deep learning operation module grayscales the surface image of the detection object captured by the image capture module using the semantic segmentation algorithm, and performs subsequent image processing. A large amount of learning data is not required in the process. In this embodiment, the user chooses to use supervised learning. Supervised learning requires labeled training data, that is, images containing correct surface defect labels. During the training process, the image capture module is used to capture different angles of the weld surface without defects. For the weld defect detection object, the image capture module is used to capture the surface defect image, and the defect area is marked. The cold crack defect image is captured and marked as a cold crack, and then the undercut defect image is captured and marked as an undercut, and then the pore defect image is captured and marked as a pore. By analogy, two sets of training data with and without weld defects are finally obtained and transmitted to the intelligent data module. In addition, the intelligent data module automatically obtains web page data to obtain weld defect image data and non-weld defect image data and stores them in the intelligent database to provide different types of weld defect image data and non-weld defect image data for deep learning calculation module training. The deep learning algorithm will compare the weld defect surface and non-defect image of the training data from each pixel, and then perform image processing, semantic segmentation operation and normalization processing to output the detection results.
[0104] During the inspection process, first, after each weld is completed, the welder will use the two-dimensional visible light camera 8 to shoot the weld effect. The configurable parameters of the two-dimensional visible light camera include shooting frequency, focal length, exposure, white balance, contrast, and saturation. The deep learning operation module automatically calculates and optimizes the parameters in a closed loop, and sets the parameters to the best clarity, resolution, and light and dark environment, so that they can be fed back to control the parameters of the two-dimensional visible light camera. All two-dimensional visible light cameras synchronously correct the parameters of the two-dimensional visible light camera, so that the deep learning operation module can calculate accurate results. The outer ring of the two-dimensional visible light camera is equipped with a ring light source, which is composed of dense light-emitting diodes and connected to the deep learning operation module. The brightness of each light-emitting diode can be independently controlled by the deep learning operation module. The actual brightness of the weld will be sensed by the photosensitive element of the two-dimensional visible light camera. If the brightness of the concave part in the weld is low, the signal will be transmitted to the deep learning operation module. The deep learning operation module determines that the actual brightness of the concave part in the weld is dark, so that the image capture module cannot clearly capture all the contours of the detection object. The deep learning operation module will control the light emitting diode of the ring light source closest to the concave part in the weld to increase the brightness, so that the two-dimensional visible light camera can capture a high-resolution image. This image capture module can be applied to different scenarios, and the number of image capture modules and the design of the fixture for fixing the image capture are not limited by this patent.
[0105] The two-dimensional visible light camera takes pictures of the surface of the weld. The deep learning operation module uses image processing to calculate the weld image with a function algorithm and judge whether each pixel is noise, and filters the noise. The function algorithm includes but is not limited to the median filter algorithm, the mean filter algorithm, the Gaussian filter algorithm, the wavelet noise reduction algorithm, the low-rank matrix recovery algorithm, etc., which reduces the noise points and noise in the image, improves the quality and clarity of the image, and outputs high-quality and clear images to the next convolution layer. After image processing, the image obtains an image without surface noise, and then semantic segmentation is performed to obtain a grayscale image, and then normalization is performed to obtain a feature map. Finally, the deep learning operation module outputs the detection result according to the feature map. For example, the probability of 3 consecutive pixels on the central surface of the weld is 1, and the final output result is a defect image to the monitoring platform. The monitoring platform displays the analysis results of the deep learning operation module on the surface of the detection object in real time on the monitoring platform, showing the defect result, the defect type is a pore defect, the defect position is the central position, the defect area is 3 mm, and the number of defects is 1.
[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A surface defect detection model training method, characterized in that: The method comprises: Obtain a training sample image set; Input the training sample images into the deep learning-based surface defect detection model; The surface defect detection model based on deep learning performs semantic segmentation on the input training sample image to obtain the original feature map, and normalizes the original feature map to obtain the normalized feature map; The surface defect detection model based on deep learning is trained using the normalized feature map to obtain a trained surface defect detection model based on deep learning.
2. The surface defect detection model training method according to claim 1, characterized in that: Before the step of semantically segmenting the surface image of the training sample image to obtain the original feature map, the following step is also included: performing denoising on the surface image of the training sample image.
3. The surface defect detection model training method according to claim 1, characterized in that: The formula for normalizing the original feature map is: P(pixel) = (D(pixel)-D(min)) / (D(max)-D(min)), where: P(pixel) is the degree of defect of the pixel, D(pixel) is the measurement value of the pixel on the preset feature, D(min) is the minimum measurement value of all pixels in the entire image on the preset feature, and D(max) is the maximum measurement value of all pixels in the entire image on the preset feature.
4. The surface defect detection model training method according to claim 1, characterized in that: The training sample image set includes a first training sample image set and a second training sample image set. The first training sample image set includes a number of images without surface defects, and the second training sample image set includes a number of images with surface defects. The images with surface defects include marked surface defect images and / or unmarked surface defect images.
5. The surface defect detection model training method according to claim 1, characterized in that: The training sample image set is obtained by shooting with an image capture module; and / or, The training sample image set is obtained by automatically acquiring web page data; The steps of obtaining a training sample image set by automatically acquiring web page data include: Obtain HTML content: Send HTTP request to the server of the target website to obtain HTML content of scientific research web pages or other web pages related to surface defects; Parsing HTML: After obtaining the HTML content, parse it to extract the required information; Select and extract data: Use selectors to locate specific HTML elements that contain the required data, and then extract image database information from these elements.
6. A surface defect detection method, characterized in that: The steps include: Acquire a surface image of the object to be measured; The surface image of the object to be tested is input into a surface defect detection model based on deep learning obtained by training using the surface defect detection model training method as described in any one of claims 1 to 5, and the surface defect detection model based on deep learning outputs a defect detection result.
7. The surface defect detection method according to claim 6, characterized in that: The surface image of the object to be measured is obtained by an image capture module, wherein the image capture module includes a camera with an annular light source in the outer ring. Before the surface image of the object to be measured is obtained by the image capture module, the method further includes: optimizing the parameters of the image capture module; Optimizing the parameters of the image capture module specifically includes: receiving a surface image of the object to be measured taken by the image capture module, and obtaining the current brightness of each light source and the parameters of the camera, analyzing the received surface image, and determining whether the clarity of all contours of the object to be measured meets the requirements; if not, feedback is provided to adjust the brightness of each light source of the ring light source and the parameters of the camera until the surface image of the object to be measured taken by the image capture module meets the requirements, wherein the parameters of the camera include at least one of shooting frequency, focal length, exposure, white balance, contrast, and saturation.
8. A surface defect detection system, characterized in that: include: An image capturing module, wherein the image capturing module is used to capture all surfaces of the object to be measured to obtain a surface image of the object to be measured; A deep learning operation module, wherein the deep learning operation module is provided with a surface defect detection model based on deep learning obtained by training using the surface defect detection model training method as described in any one of claims 1 to 5, and the surface defect detection model based on deep learning is used to receive a surface image of an object to be tested and output a defect detection result.
9. The surface defect detection system according to claim 8, characterized in that: It also includes a monitoring platform, which is used to display the defect detection results output by the deep learning operation module; The defect detection result includes a normal result or a defect result, and when the defect detection result is a defect result, the defect detection result also includes: at least one of: defect type, defect position, defect area, and defect number; It also includes an intelligent data module, which is used to obtain and store the training sample image set taken by the image capture module, and / or the intelligent data module is used to obtain and store the training sample image set by automatically acquiring web page data.
10. The surface defect detection system according to claim 8, characterized in that: The image capture module includes a camera with an annular light source in the outer ring, the annular light source includes a number of light-emitting bodies distributed around the camera, each light-emitting body is connected to a deep learning operation module, and the brightness of each light-emitting body is independently controlled by the deep learning operation module. The deep learning operation module is used to receive the surface image of the object to be measured taken by the image capture module, and obtain the current brightness of each light-emitting body and the parameters of the camera. The deep learning operation module is also used to analyze the received surface image to determine whether the clarity of all contours of the object to be measured meets the requirements. If not, the brightness of each light-emitting body of the annular light source and the parameters of the camera are adjusted by feedback until the surface image of the object to be measured taken by the image capture module meets the requirements. The parameters of the camera include at least one of shooting frequency, focal length, exposure, white balance, contrast, and saturation.