Defect detection system based on intelligent camera information extraction

Abstract

The invention relates to the technical field of flaw detection, in particular to a flaw detection system based on intelligent camera information extraction, which comprises the following components: a high-definition intelligent camera, an image acquisition card, an FPGA (Field Programmable Gate Array) core processor and automatic control equipment. The flaw detection system based on intelligent camera information extraction is provided with a yolov8 detection module, a C3 module in YOLOv5 is replaced by a C2f module, more jump layer connections and additional Split operations are added, the utilization efficiency of feature information is improved, deep learning model training in an FPGA core processor is arranged, and based on a large number of marked flaw image data, the detection accuracy is improved. And training the deep learning model to enable the deep learning model to have a defect identification capability, inputting an image of a to-be-detected product into the trained deep learning model, and carrying out defect detection and classification, so that the system can cope with complex and variable defect types.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and in particular to a defect detection system based on intelligent camera information extraction. Background Art

[0002] In the manufacturing industry, defect detection is a crucial link, which is directly related to product quality and customer satisfaction. Traditional manual inspection methods are not only inefficient, but also easily affected by human factors, resulting in inconsistent inspection results and a high rate of missed detection. Therefore, the development of an efficient, accurate and automated defect detection system has become an urgent need in the industry.

[0003] The prior art discloses a machine vision-based intelligent defect recognition system in sheet building material manufacturing with publication number: CN112098428A. The technical solution of the present invention is: an intelligent defect recognition system based on machine vision in sheet building material manufacturing, comprising: a sheet building material conveying mechanism; a standard industrial camera located above the sheet building material conveying mechanism, used to collect product images of sheet building materials on the sheet building material conveying mechanism; a light source, used to illuminate the sheet building materials placed on the sheet building material conveying mechanism by the standard industrial camera; a sensor trigger, used to notify the standard industrial camera to obtain the sheet building material image when the sheet building material passes the position directly below the standard industrial camera; a processing unit, used to obtain the product image collected by the standard industrial camera, and intelligently recognize the product image, analyze the detection result of the sheet building material, obtain the product image in real time through the standard industrial camera, transmit it to the processing unit, perform real-time image analysis and processing, identify various defects on the surface of the tile and intelligently identify the category, accurately measure the size of the defect through the image, and determine the grade of the product and give a mark.

[0004] However, the above-mentioned defect detection system uses a standard industrial camera to obtain product images in real time, transmits them to the processing unit, performs real-time image analysis and processing, identifies various defects on the surface of the tile and intelligently identifies the categories. However, the above-mentioned device does not introduce a deep learning model to perform feature extraction and defect identification on the image data collected by the camera. It cannot cope with complex and changeable defect types and has poor practicality. Therefore, it is very necessary to design a practical defect detection system based on intelligent camera information extraction. Summary of the invention

[0005] The object of the present invention is to provide a defect detection system based on intelligent camera information extraction to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a defect detection system based on intelligent camera information extraction, the defect detection system includes the following components:

[0007] High-definition smart cameras, image acquisition cards, FPGA core processors and automation control equipment.

[0008] High-definition smart camera: located above the automation control equipment, it is used to collect images of the products to be inspected transmitted from the automation control equipment. The high-definition smart camera contains a camera integrated processor and a yolov8 detection module.

[0009] Image acquisition card: controls the camera to acquire images and transmits the acquired image data to the processing unit. The image acquisition card contains an image preprocessing module.

[0010] FPGA core processor: As an image processing unit, it is responsible for the preprocessing and high-speed processing of image data. The FPGA core processor includes a data analysis and display module.

[0011] Automatic control equipment: A conveyor belt control system is set up, and a rejection component is set above the conveyor belt control system to realize automatic rejection or marking of defective products.

[0012] Preferably, the FPGA core processor is provided with deep learning model training, and the deep learning model is trained based on a large amount of labeled defect image data so that it has the ability to identify defects, and the image of the product to be inspected is input into the trained deep learning model for defect detection and classification.

[0013] Preferably, the smart camera, image acquisition card, FPGA core processor and automation control equipment are integrated into the defect detection system, and the system is integrated with the production line to achieve real-time defect detection and control in the production process.

[0014] Preferably, the smart camera, image acquisition card, FPGA core processor and automation control equipment are integrated into the defect detection system, and the system is integrated with the production line to achieve real-time defect detection and control in the production process.

[0015] Preferably, the yolov8 detection module introduces the DFL loss function as part of the regression loss in Loss calculation, combined with the loss function CIOU Loss, to jointly improve the convergence speed and detection accuracy of the model.

[0016] Preferably, the basic structure of the yolov8 detection module is in the form of backbone+neck+head, wherein:

[0017] The Backbone module is responsible for extracting feature information from the image. The Backbone module draws on the CSPDarkNet structure network structure, but replaces the C3 module in YOLOv5 with the C2f module, adds more skip-layer connections and additional Split operations, which helps to improve feature extraction capabilities.

[0018] Preferably, the Neck module in the yolov8 detection module is responsible for further fusing and enhancing feature information. The Neck module adopts the PANet structure, and by introducing a bottom-up path, it makes it easier to transfer the bottom-level information to the top-level, thereby improving the utilization efficiency of the feature information.

[0019] Preferably, the Head module in the yolov8 detection module is responsible for outputting the location and category information of the target. The Head module part adopts a decoupled head structure to separate the classification and detection heads, and uses the integral form representation proposed in the Distribution Focal Loss strategy for the regression branch to improve the detection accuracy.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] 1. The defect detection system based on intelligent camera information extraction is equipped with a deep learning model training in the FPGA core processor. Based on a large amount of labeled defect image data, the deep learning model is trained to enable it to have the ability to identify defects. The image of the product to be inspected is input into the trained deep learning model to perform defect detection and classification, so that the system can cope with complex and changeable defect types.

[0022] 2. This defect detection system based on intelligent camera information extraction is equipped with a yolov8 detection module, which replaces the C3 module in YOLOv5 with a C2f module, adds more skip-layer connections and additional Split operations, which helps to improve the feature extraction capability. It adopts the PANet structure and introduces a bottom-up path to make it easier for the bottom-level information to be transmitted to the top level, thereby improving the utilization efficiency of feature information.

[0023] 3. This defect detection system based on intelligent camera information extraction separates the classification and detection heads by setting up the Head module in the yolov8 detection module, and uses the integral form representation proposed in the Distribution Focal Loss strategy for the regression branch to improve the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0025] Figure 1 It is a schematic diagram of the defect detection system of the present invention;

[0026] Figure 2 It is a schematic diagram of the module workflow in the defect detection system of the present invention; DETAILED DESCRIPTION

[0027] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0028] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0029] See also Figure 1-2 , the present invention provides the following technical solutions:

[0030] A defect detection system based on intelligent camera information extraction, the defect detection system includes the following components: a high-definition intelligent camera, an image acquisition card, an FPGA core processor and an automatic control device.

[0031] High-definition smart camera: located above the automation control equipment, it is used to collect images of the products to be inspected transmitted from the automation control equipment. The high-definition smart camera contains a camera integrated processor and a yolov8 detection module.

[0032] Image acquisition card: controls the camera to acquire images and transmits the acquired image data to the processing unit. The image acquisition card contains an image preprocessing module.

[0033] FPGA core processor: As an image processing unit, it is responsible for the preprocessing and high-speed processing of image data. The FPGA core processor includes a data analysis and display module.

[0034] Automatic control equipment: A conveyor belt control system is set up, and a rejection component is set above the conveyor belt control system to realize automatic rejection or marking of defective products.

[0035] The FPGA core processor is equipped with deep learning model training. Based on a large amount of labeled defect image data, the deep learning model is trained to enable it to have the ability to identify defects. The image of the product to be inspected is input into the trained deep learning model for defect detection and classification.

[0036] Smart cameras, image acquisition cards, FPGA core processors and automated control devices are integrated into the defect detection system, which is integrated with the production line to achieve real-time defect detection and control during the production process.

[0037] Smart cameras, image acquisition cards, FPGA core processors and automated control devices are integrated into the defect detection system, which is integrated with the production line to achieve real-time defect detection and control during the production process.

[0038] The yolov8 detection module introduces the DFL loss function as part of the regression loss in terms of Loss calculation, and combines it with the loss function CIOU Loss to jointly improve the convergence speed and detection accuracy of the model.

[0039] The basic structure of the yolov8 detection module is in the form of backbone+neck+head, where:

[0040] The Backbone module is responsible for extracting feature information from the image. The Backbone module draws on the CSPDarkNet structure network structure, but replaces the C3 module in YOLOv5 with the C2f module, adds more skip-layer connections and additional Split operations, which helps to improve feature extraction capabilities.

[0041] The Neck module in the yolov8 detection module is responsible for further fusing and enhancing feature information. The Neck module adopts the PANet structure. By introducing a bottom-up path, it makes it easier to pass the bottom-level information to the top level, thereby improving the utilization efficiency of feature information.

[0042] The Head module in the yolov8 detection module is responsible for outputting the location and category information of the target. The Head module adopts a decoupled head structure to separate the classification and detection heads, and uses the integral form representation proposed in the Distribution Focal Loss strategy for the regression branch to improve the detection accuracy.

[0043] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0044] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A defect detection system based on intelligent camera information extraction, characterized in that: The defect detection system consists of the following components: High-definition smart cameras, image acquisition cards, FPGA core processors and automation control equipment; High-definition smart camera: located above the automation control equipment, used to collect images of the products to be tested transmitted from the automation control equipment. The high-definition smart camera contains a camera integrated processor and a yolov8 detection module; Image acquisition card: controls the camera to acquire images and transmits the acquired image data to the processing unit. The image acquisition card contains an image preprocessing module; FPGA core processor: As an image processing unit, it is responsible for the preprocessing and high-speed processing of image data. The FPGA core processor includes a data analysis and display module; Automatic control equipment: A conveyor belt control system is set up, and a rejection component is set above the conveyor belt control system to realize automatic rejection or marking of defective products.

2. The defect detection system based on intelligent camera information extraction according to claim 1, characterized in that: The FPGA core processor is provided with deep learning model training. Based on a large amount of labeled defect image data, the deep learning model is trained to enable it to have the ability to identify defects. The image of the product to be inspected is input into the trained deep learning model to perform defect detection and classification.

3. The defect detection system based on intelligent camera information extraction according to claim 2, characterized in that: The smart camera, image acquisition card, FPGA core processor and automatic control equipment are integrated into the defect detection system, and the system is integrated with the production line to realize real-time defect detection and control in the production process.

4. The defect detection system based on intelligent camera information extraction according to claim 3 is characterized in that: A new backbone network is introduced in the YOLOv8 detection module. The design concept of YOLOv7ELAN is referred to. The C3 structure in YOLOv5 is replaced with a C2f structure with richer gradient flow. Different numbers of channels are adjusted for models of different scales to achieve further lightweighting.

5. The defect detection system based on intelligent camera information extraction according to claim 4, characterized in that: The yolov8 detection module introduces the DFL loss function as part of the regression loss in terms of loss calculation, combined with the loss function CIOU Loss to jointly improve the convergence speed and detection accuracy of the model.

6. The defect detection system based on intelligent camera information extraction according to claim 5, characterized in that: The basic structure of the yolov8 detection module is in the form of backbone+neck+head, where: The Backbone module is responsible for extracting feature information from the image. The Backbone module draws on the CSPDarkNet structure network structure, but replaces the C3 module in YOLOv5 with the C2f module, adds more skip-layer connections and additional Split operations, which helps to improve feature extraction capabilities.

7. The defect detection system based on intelligent camera information extraction according to claim 6, characterized in that: The Neck module in the yolov8 detection module is responsible for further fusing and enhancing feature information. The Neck module adopts the PANet structure. By introducing a bottom-up path, the bottom-level information is more easily transmitted to the top level, thereby improving the utilization efficiency of the feature information.

8. The defect detection system based on intelligent camera information extraction according to claim 7, characterized in that: The Head module in the yolov8 detection module is responsible for outputting the location and category information of the target. The Head module adopts a decoupled head structure to separate the classification and detection heads, and uses the integral form representation proposed in the Distribution Focal Loss strategy for the regression branch to improve the detection accuracy.

Images