Industrial defect real-time detection and positioning method, system and device and medium

Through deep neural network hybrid model and optimization technology, the problem that traditional methods are difficult to detect complex, small or hidden defects is solved, efficient and accurate detection and positioning of industrial defects is achieved, and automation level and production efficiency are improved.

CN118967627APending Publication Date: 2024-11-15INSPUR YUNZHOU (SHANDONG) IND INTERNET CO LTD

Patent Information

Application Number
CN202411058799.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Traditional industrial defect detection methods, such as manual visual inspection and rule-based image analysis, are difficult to detect and locate complex, small or hidden defects in real time and accurately, resulting in difficult to guarantee detection efficiency and accuracy and high cost.

Method used

Adopting a hybrid deep neural network model, combining convolutional neural network and recurrent neural network, the attention mechanism, border regression and key point estimation algorithm are introduced, and the model is optimized to achieve real-time detection and positioning of industrial defects through self-supervised pre-training and transfer learning. , and improve inference capabilities through model lightweighting and hardware acceleration technologies.

Benefits of technology

It realizes efficient and accurate industrial defect detection and positioning, improves automation level and production efficiency, reduces defective yield rates, simplifies operating procedures, and improves the stability and reliability of the system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967627A_ABST
    Figure CN118967627A_ABST
Patent Text Reader

Abstract

The invention provides an industrial defect real-time detection and positioning method, system and device and a medium, and the method comprises the steps: collecting an industrial defect data image, and carrying out the preprocessing of the industrial defect data image; marking the industrial defect data image to generate an industrial defect detection data set; fusing the convolutional neural network and the recurrent neural network to generate a deep neural network hybrid model; performing self-supervised pre-training and transfer learning according to the deep neural network hybrid model and the industrial defect detection data set; introducing an attention mechanism for the deep neural network hybrid model, and outputting coordinate information of a defect area in combination with frame regression and a key point estimation algorithm; optimizing the model by adopting a model lightweight technology and a hardware acceleration technology; and packaging the model and a corresponding algorithm into a preset industrial detection system, and carrying out real-time detection and positioning of industrial defects through linkage with production line equipment. According to the invention, defects of industrial products can be efficiently and accurately detected and positioned.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computer vision and artificial intelligence technology, and more specifically to a method, system, device and medium for real-time detection and positioning of industrial defects. Background Art

[0002] In the modern industrial production system, product quality is not only the core element of corporate competitiveness, but also the key to consumer rights, brand image and even social public safety. Traditional quality control methods, such as manual visual inspection and rule-based image analysis methods, to a certain extent, bear the important responsibility of ensuring product quality. However, with the continuous expansion of industrial production scale, the increasing complexity of products and the increasingly stringent market requirements for product quality, these traditional methods have exposed many limitations when faced with complex, small or hidden manufacturing defects.

[0003] Since manual visual inspection relies heavily on the operator's experience, concentration, and visual acuity, it is easily affected by individual differences, fatigue and other factors, resulting in frequent missed detections and false detections, and it is difficult to ensure detection efficiency and accuracy. Especially in large-scale, high-speed production environments, manual visual inspection is not only time-consuming and labor-intensive, but also has high labor costs, becoming an important bottleneck restricting the improvement of production efficiency and economic benefits. Although the rule-based image analysis method has realized the automation of the detection process to a certain extent, its rule setting is often based on the prediction of typical defect patterns, and its ability to identify defects with complex morphology and strong variability is limited. When dealing with small or hidden defects, this type of method may make misjudgments or missed judgments due to the limitations of image feature extraction and recognition algorithms, and the rule update lags behind product process changes, making it difficult to adapt to new defect types brought about by product iteration and upgrades.

[0004] As current industrial production places increasingly higher demands on quality control methods, how to accurately detect and locate complex, tiny or hidden defects in various industrial products in real time has become an urgent problem that we need to solve. Summary of the invention

[0005] In response to the above problems, the purpose of the present invention is to provide a method, system, device and medium for real-time detection and positioning of industrial defects, which can efficiently and accurately detect and locate defects in industrial products, improve the automation level and production efficiency of industrial quality inspection, and reduce the defective product rate.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0007] In a first aspect, the present invention discloses a method for real-time detection and positioning of industrial defects, comprising:

[0008] Use video acquisition equipment to collect industrial defect data images and preprocess the industrial defect data images; annotate the industrial defect data images to generate industrial defect detection data sets;

[0009] Fusion of convolutional neural networks and recurrent neural networks to generate a deep neural network hybrid model with spatial and temporal scales;

[0010] Performing self-supervised pre-training and transfer learning based on the deep neural network hybrid model and the industrial defect detection dataset;

[0011] Introducing the attention mechanism into the deep neural network hybrid model, combining the bounding box regression and key point estimation algorithm to output the coordinate information of the defect area;

[0012] Use model lightweight technology and hardware acceleration technology to optimize deep neural network hybrid models;

[0013] The deep neural network hybrid model and the corresponding algorithm are encapsulated into the preset industrial inspection system, and real-time detection and positioning of industrial defects are carried out by linking with the production line equipment.

[0014] Furthermore, the method of collecting industrial defect data images using a video acquisition device and preprocessing the industrial defect data images includes:

[0015] Use high-definition cameras and industrial endoscopes to collect images of industrial products including normal samples and various defective samples;

[0016] Perform data cleaning on industrial product images.

[0017] Furthermore, the industrial defect data image is annotated to generate an industrial defect detection data set, including:

[0018] By labeling the industrial defect data images by category, the crack defects, dent defects and scratch defects in them are marked;

[0019] By annotating the locations of industrial defect data images, the bounding boxes and key points of industrial product images are determined.

[0020] Furthermore, the convolutional neural network and the recurrent neural network are integrated to generate a deep neural network hybrid model with spatial scale and time scale, including:

[0021] Use a convolutional neural network to extract features from each frame of industrial product images and generate a feature sequence. Use the feature sequence extracted by the convolutional neural network as the input of the recurrent neural network, capture the evolution and dependency of the feature sequence in the time dimension through the recurrent neural network, and learn the dynamic behavior of the recurrent neural network.

[0022] After the recurrent neural network outputs the updated feature sequence, further feature integration is performed by connecting the fully connected layer, and a classifier or regressor is used to complete the spatiotemporal detection and identification of industrial defects.

[0023] Further, the self-supervised pre-training and transfer learning are performed according to the deep neural network hybrid model and the industrial defect detection data set, including:

[0024] Extract unlabeled or weakly labeled industrial defect data images from the industrial defect detection dataset;

[0025] Using unlabeled or weakly labeled industrial defect data images for self-supervised pre-training to enhance the learning of common industrial texture and edge features by the deep neural network hybrid model;

[0026] Transfer learning is performed based on the deep neural network hybrid model to adapt it to the defect detection task.

[0027] Furthermore, the attention mechanism is introduced into the deep neural network hybrid model, and the coordinate information of the defect area is output by combining the border regression and key point estimation algorithm, including:

[0028] By introducing the attention mechanism in the middle layer of the deep neural network hybrid model, an attention weight is generated for each position of the industrial product image;

[0029] At the back end of the deep neural network hybrid model, combined with the bounding box regression or key point estimation method, the coordinate information of the defect area is output according to the recognition result of the defect area by the deep neural network hybrid model.

[0030] Furthermore, the use of model lightweight technology and hardware acceleration technology to optimize the deep neural network hybrid model includes:

[0031] Improve the reasoning and computing capabilities of deep neural network hybrid models through channel pruning, model quantization, and knowledge distillation technologies;

[0032] The computational graph of the deep neural network hybrid model is optimized and mapped to the AI ​​hardware for processing.

[0033] In a second aspect, the present invention further discloses a real-time industrial defect detection and positioning system, comprising: a data acquisition module, for acquiring industrial defect data images using a video acquisition device, and preprocessing the industrial defect data images;

[0034] The data set construction module is used to annotate industrial defect data images and generate industrial defect detection data sets;

[0035] A model building module for fusing convolutional neural networks with recurrent neural networks to generate a deep neural network hybrid model with spatial and temporal scales;

[0036] A model training module, used for performing self-supervised pre-training and transfer learning based on the deep neural network hybrid model and the industrial defect detection dataset;

[0037] The defect area recognition module is used to introduce the attention mechanism into the deep neural network hybrid model, combine the bounding box regression and key point estimation algorithm, and output the coordinate information of the defect area;

[0038] Model optimization module, which is used to optimize the deep neural network hybrid model using model lightweight technology and hardware acceleration technology;

[0039] The linkage detection module is used to encapsulate the deep neural network hybrid model and the corresponding algorithm into the preset industrial detection system, and perform real-time detection and positioning of industrial defects by linkage with production line equipment.

[0040] In a third aspect, the present invention further discloses a real-time detection and positioning device for industrial defects, comprising:

[0041] A memory for storing a program for real-time detection and location of industrial defects;

[0042] A processor is used to implement the steps of the method for real-time industrial defect detection and positioning as described in any one of the above items when executing the real-time industrial defect detection and positioning program.

[0043] In a fourth aspect, the present invention further discloses a readable storage medium, on which is stored a program for real-time detection and positioning of industrial defects. When the program for real-time detection and positioning of industrial defects is executed by a processor, the steps of the method for real-time detection and positioning of industrial defects as described in any one of the above items are implemented.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention discloses a method, system, device and medium for real-time detection and positioning of industrial defects. By integrating advanced deep learning technology and optimization means, efficient and accurate real-time detection and positioning of industrial defects are achieved. The present invention not only has the advantages of real-time and accuracy, but also further improves the accuracy and efficiency of defect detection by introducing attention mechanism and coordinate information output function. At the same time, the present invention adopts model lightweight and hardware acceleration technology to optimize the reasoning ability of the model so that it can meet the requirements of industrial real-time processing. Finally, the present invention realizes the automation and intelligence of industrial defect detection through system integration and application, providing strong technical support for industrial production.

[0046] The present invention constructs a deep neural network hybrid model with spatial and temporal scales by fusing convolutional neural networks and recurrent neural networks and combining attention mechanisms. This model can accurately capture the defect features in industrial product images and output the precise coordinate information of the defect area in real time through border regression and key point estimation algorithms. This efficient defect detection and positioning capability greatly improves the automation level of industrial production and product quality.

[0047] In order to improve the reasoning speed of the model and reduce the consumption of computing resources, the present invention adopts model lightweight technologies, such as channel pruning, model quantization and knowledge distillation, which effectively reduce the size and complexity of the model. At the same time, combined with computational graph optimization and hardware acceleration technology, the optimized model is mapped to dedicated AI hardware for processing, further improving the reasoning performance and real-time performance of the model. These optimization measures enable the model to maintain high performance while being more adaptable to the real-time processing needs of industrial production.

[0048] The present invention encapsulates the optimized deep neural network hybrid model and the corresponding algorithm into a preset industrial detection system, achieving seamless docking and linkage with production line equipment. This integrated solution not only simplifies the operation process and reduces the skill requirements for operators, but also improves the stability and reliability of the system. In addition, through real-time feedback of defect detection results, enterprises can adjust production processes in a timely manner, optimize product quality control, and realize intelligent management of the production process.

[0049] It can be seen that compared with the prior art, the present invention has outstanding substantive features and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0051] Figure 1 It is a method flow chart of a specific implementation mode of the present invention.

[0052] Figure 2 It is a system structure diagram of a specific implementation mode of the present invention.

[0053] Figure 3 It is a device structure diagram of a specific implementation mode of the present invention.

[0054] In the figure, 1. data acquisition module; 2. data set construction module; 3. model construction module; 4. model training module; 5. defect area identification module; 6. model optimization module; 7. linkage detection module; 101. processor; 102. memory; 103. input interface; 104. output interface; 105. communication unit; 106. keyboard; 107. display; 108. mouse. DETAILED DESCRIPTION

[0055] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0056] See also Figure 1 As shown, this embodiment provides a method for real-time detection and positioning of industrial defects, including the following steps:

[0057] S1: Use video acquisition equipment to collect industrial defect data images and preprocess the industrial defect data images.

[0058] In a specific implementation, first, high-definition cameras and industrial endoscopes are used to collect industrial product images containing normal samples and various defect samples; then, the industrial product images are cleaned and processed to remove low-quality images such as blur, uneven lighting, and occlusion that affect model training.

[0059] S2: Label the industrial defect data images and generate an industrial defect detection dataset.

[0060] In a specific implementation, the industrial defect data image is annotated by category to mark the crack defects, dent defects and scratch defects therein; the industrial defect data image is annotated by position to determine the bounding box and key points of the industrial product image.

[0061] S3: Fusion of convolutional neural networks and recurrent neural networks to generate a deep neural network hybrid model with spatial and temporal scales.

[0062] In a specific embodiment, first, a convolutional neural network (CNN) is used to extract features from the input of each frame or each moment. CNN captures spatial features such as the shape, color and texture of the defect through convolution operations, and performs dimensionality reduction and spatial invariance extraction through a pooling layer. A feature sequence is generated based on the extracted features.

[0063] Then, the feature sequence extracted by CNN is used as the input of recurrent neural network (RNN). RNN can adopt LSTM or GRU. RNN can capture the evolution and dependency of feature sequence in time dimension and learn the long-term dynamic behavior of feature sequence.

[0064] Finally, after the RNN output, the fully connected layer (FCN) is connected for further feature integration, and finally the classifier or regressor is connected to complete the spatiotemporal detection and recognition of industrial defects. The combination of the two improves the recognition ability of complex and continuous defects.

[0065] S4: Perform self-supervised pre-training and transfer learning based on the deep neural network hybrid model and the industrial defect detection dataset.

[0066] In a specific implementation, a large number of unlabeled or weakly labeled industrial defect data images are first extracted from the industrial defect detection dataset. Then, the unlabeled or weakly labeled industrial defect data images are used for self-supervised pre-training, and tasks such as mask prediction and contrastive learning can be used to enhance the learning of general industrial texture and edge features by the deep neural network hybrid model. Finally, transfer learning is performed based on the deep neural network hybrid model to enable it to quickly adapt to defect detection tasks in specific industries.

[0067] S5: Introduce the attention mechanism into the deep neural network hybrid model, combine the bounding box regression and key point estimation algorithm, and output the coordinate information of the defect area.

[0068] In a specific implementation, first, an attention mechanism (Transformer Self-attention) is introduced to weight the features of the middle layer of the deep neural network hybrid model, highlight the potential defect area, and assist the model to accurately locate the defect position. Specifically, the Transformer self-attention mechanism is introduced in the middle layer of the deep neural network hybrid model, and an attention weight is generated for each position by calculating the correlation between different positions in the feature map. These weights are used to weight the feature map to highlight the potential defect area. This weighted processing helps the model to locate the defect position more accurately and improve the accuracy of defect detection.

[0069] Then, combined with bounding box regression or key point estimation methods, the precise coordinate information of the defect is output. Specifically, at the back end of the deep neural network hybrid model, combined with bounding box regression or key point estimation methods, the precise coordinate information of the defect is output based on the model's recognition results of the defect area. The bounding box regression method locates the defect area by predicting the coordinates of the upper left and lower right corners of the bounding box, while the key point estimation method locates the defect by predicting the coordinates of the key points (such as center points, corner points, etc.) of the defect area. Both methods can provide specific location information of the defect in the image, helping users to quickly and accurately identify and handle defects.

[0070] S6: Use model lightweight technology and hardware acceleration technology to optimize deep neural network hybrid models.

[0071] In a specific implementation, firstly, the reasoning computing capability of the deep neural network hybrid model is improved through model lightweight technologies such as channel pruning, model quantization, and knowledge distillation. Specifically, in order to improve the reasoning speed of the model and reduce the consumption of computing resources, a series of model lightweight technologies are adopted. These technologies include channel pruning (reducing the number of channels of the model), model quantization (converting the weights and activation values ​​of the model from floating point numbers to fixed point numbers), and knowledge distillation (using a powerful teacher model to guide a small student model for training). Through these methods, the size and computational complexity of the model are reduced, thereby improving the reasoning computing capability and making it more suitable for operation in resource-constrained environments.

[0072] Then, the calculation graph is optimized and deeply integrated with dedicated AI hardware (such as GPU, FPGA, ASIC) to ensure that the system runs stably under the real-time processing speed requirements of the production line. Specifically, in order to meet the real-time processing speed requirements of the production line, the calculation graph of the deep neural network hybrid model is optimized and deeply integrated with dedicated AI hardware (such as GPU, FPGA, ASIC). The calculation graph optimization reduces memory access and computational redundancy by rearranging and merging calculation steps, thereby improving execution efficiency. The deep integration with dedicated AI hardware utilizes the powerful computing power of these hardware to map the optimized calculation graph to the hardware for execution, further improving the real-time processing speed and stability of the system. This method of hardware and software collaborative optimization ensures that the system can run stably on the production line and meet the requirements of real-time processing.

[0073] S7: Encapsulate the deep neural network hybrid model and the corresponding algorithm into the preset industrial inspection system, and perform real-time detection and positioning of industrial defects by linking with the production line equipment.

[0074] In a specific implementation, the deep neural network hybrid model and the corresponding algorithm are encapsulated in a complete industrial inspection system, including functional modules such as image acquisition, preprocessing, defect detection, result display and alarm, data recording and analysis, etc. The system is linked with the production line equipment to form a closed-loop control, and the production process parameters are adjusted in real time according to the inspection results to prevent defects.

[0075] When performing real-time detection and positioning of industrial defects, a deep neural network hybrid model is used to implement industrial defect detection. If a defect is identified, the defect is immediately displayed and alarmed, and the relevant information is stored in the database.

[0076] The present invention provides a method for real-time detection and positioning of industrial defects, which encapsulates the optimized deep neural network hybrid model and the corresponding algorithm into a preset industrial detection system, and realizes seamless docking and linkage with production line equipment. This integrated solution not only simplifies the operation process and reduces the skill requirements for operators, but also improves the stability and reliability of the system. In addition, through the real-time feedback of defect detection results, enterprises can adjust the production process in time, optimize product quality control, and realize intelligent management of the production process.

[0077] See also Figure 2 As shown, the present invention also discloses a real-time industrial defect detection and positioning system, including: a data acquisition module 1, a data set construction module 2, a model construction module 3, a model training module 4, a defect area identification module 5, a model optimization module 6 and a linkage detection module 7.

[0078] The data acquisition module 1 is used to acquire industrial defect data images using a video acquisition device and pre-process the industrial defect data images.

[0079] The data set construction module 2 is used to annotate the industrial defect data images and generate an industrial defect detection data set.

[0080] Model building module 3 is used to fuse convolutional neural network and recurrent neural network to generate a deep neural network hybrid model with spatial scale and time scale.

[0081] The model training module 4 is used to perform self-supervised pre-training and transfer learning based on the deep neural network hybrid model and the industrial defect detection data set.

[0082] The defect area recognition module 5 is used to introduce an attention mechanism into the deep neural network hybrid model, combine the border regression and key point estimation algorithm, and output the coordinate information of the defect area.

[0083] The model optimization module 6 is used to optimize the deep neural network hybrid model by using model lightweight technology and hardware acceleration technology.

[0084] The linkage detection module 7 is used to encapsulate the deep neural network hybrid model and the corresponding algorithm into a preset industrial detection system, and to perform real-time detection and positioning of industrial defects by linkage with production line equipment.

[0085] The specific implementation of the industrial defect real-time detection and positioning system of this embodiment is basically the same as the specific implementation of the above-mentioned industrial defect real-time detection and positioning method, and will not be repeated here.

[0086] See also Figure 3 As shown, the present invention also discloses a real-time industrial defect detection and positioning device, including a processor 101 and a memory 102; wherein, when the processor 101 executes the real-time industrial defect detection and positioning program stored in the memory, the steps of the real-time industrial defect detection and positioning method as described in any one of the above items are implemented.

[0087] The industrial defect real-time detection and positioning device provided in this embodiment may include but is not limited to a smart phone, a tablet computer, a laptop computer or a desktop computer.

[0088] Among them, the processor 101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 101 can be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 101 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 101 may be integrated with a graphics processing unit (GPU), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 101 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0089] The memory 102 may include one or more computer-readable storage media, which may be non-transitory. The memory 102 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 102 is at least used to store the following computer program, wherein, after the computer program is loaded and executed by the processor 101, the relevant steps of the method for real-time detection and positioning of industrial defects disclosed in any of the aforementioned embodiments can be implemented. In addition, the resources stored in the memory 102 may also include an operating system and data, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system may include Windows, Unix, Linux, etc. The data may include, but is not limited to, the data involved in the above-mentioned method for real-time detection and positioning of industrial defects, etc.

[0090] Furthermore, the industrial defect real-time detection and positioning device in this embodiment may also include:

[0091] The input interface 103 is used to obtain the real-time industrial defect detection and positioning program imported from the outside, and save the obtained real-time industrial defect detection and positioning program to the memory 102. It can also be used to obtain various instructions and parameters transmitted by external terminal devices, and transmit them to the processor 101, so that the processor 101 can use the above various instructions and parameters to carry out corresponding processing. In this embodiment, the input interface 103 can specifically include but is not limited to a USB interface, a serial interface, a voice input interface, a fingerprint input interface, a hard disk reading interface, etc.

[0092] The output interface 104 is used to output various data generated by the processor 101 to a terminal device connected thereto, so that other terminal devices connected to the output interface can obtain various data generated by the processor 101. In this embodiment, the output interface 104 may specifically include but is not limited to a USB interface, a serial interface, and the like.

[0093] The communication unit 105 is used to establish a remote communication connection between the server operation service optimization configuration device and the external server, so that the industrial defect real-time detection and positioning device can mount the image file to the external server. In this embodiment, the communication unit 105 may specifically include but is not limited to a remote communication unit based on wireless communication technology or wired communication technology.

[0094] The keyboard 106 is used to obtain various parameter data or instructions input by the user by tapping the keycaps in real time.

[0095] The display 107 is used to display in real time the relevant information of the real-time detection and positioning process of industrial defects.

[0096] The mouse 108 can be used to assist the user in inputting data and simplify the user's operation.

[0097] The present invention also discloses a readable storage medium, wherein the readable storage medium includes a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable hard disk, a CD-ROM or any other form of storage medium known in the technical field. The readable storage medium stores an industrial defect real-time detection and location program, and when the industrial defect real-time detection and location program is executed by a processor, the steps of the industrial defect real-time detection and location method described in any one of the above items are implemented.

[0098] In summary, the present invention can efficiently and accurately detect and locate defects in industrial products, improve the automation level and production efficiency of industrial quality inspection, and reduce the defective product rate.

[0099] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. As for the method disclosed in the embodiment, since it corresponds to the system disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0100] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0101] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0102] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0103] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each module may exist physically separately, or two or more modules may be integrated into one unit.

[0104] Similarly, each processing unit in each embodiment of the present invention may be integrated into one functional module, or each processing unit may exist physically, or two or more processing units may be integrated into one functional module.

[0105] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0106] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0107] The above is a detailed introduction to the industrial defect real-time detection and positioning method, system, device and readable storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A method for real-time detection and positioning of industrial defects, characterized in that: include: Using video acquisition equipment to collect industrial defect data images, and preprocessing the industrial defect data images; Annotate industrial defect data images to generate industrial defect detection data sets; Fusion of convolutional neural networks and recurrent neural networks to generate a deep neural network hybrid model with spatial and temporal scales; Performing self-supervised pre-training and transfer learning based on the deep neural network hybrid model and the industrial defect detection dataset; Introducing the attention mechanism into the deep neural network hybrid model, combining the bounding box regression and key point estimation algorithm to output the coordinate information of the defect area; Use model lightweight technology and hardware acceleration technology to optimize deep neural network hybrid models; The deep neural network hybrid model and the corresponding algorithm are encapsulated into the preset industrial inspection system, and real-time detection and positioning of industrial defects are carried out by linking with the production line equipment.

2. The method for real-time detection and positioning of industrial defects according to claim 1, characterized in that: The method of collecting industrial defect data images by using a video acquisition device and preprocessing the industrial defect data images includes: Use high-definition cameras and industrial endoscopes to collect images of industrial products including normal samples and various defective samples; Perform data cleaning on industrial product images.

3. The method for real-time detection and positioning of industrial defects according to claim 2, characterized in that: The step of labeling the industrial defect data image to generate an industrial defect detection data set includes: By labeling the industrial defect data images by category, the crack defects, dent defects and scratch defects in them are marked; By annotating the locations of industrial defect data images, the bounding boxes and key points of industrial product images are determined.

4. The method for real-time detection and positioning of industrial defects according to claim 2, characterized in that: The convolutional neural network and the recurrent neural network are integrated to generate a deep neural network hybrid model with spatial scale and time scale, including: Use convolutional neural networks to extract features from each frame of industrial product images and generate feature sequences; The feature sequence extracted by the convolutional neural network is used as the input of the recurrent neural network, and the evolution and dependency of the feature sequence in the time dimension are captured by the recurrent neural network, and the dynamic behavior of the recurrent neural network is learned; After the recurrent neural network outputs the updated feature sequence, further feature integration is performed by connecting the fully connected layer, and a classifier or regressor is used to complete the spatiotemporal detection and identification of industrial defects.

5. The method for real-time detection and location of industrial defects according to claim 1, characterized in that: The self-supervised pre-training and transfer learning are performed according to the deep neural network hybrid model and the industrial defect detection data set, including: Extract unlabeled or weakly labeled industrial defect data images from the industrial defect detection dataset; Using unlabeled or weakly labeled industrial defect data images for self-supervised pre-training to enhance the learning of common industrial texture and edge features by the deep neural network hybrid model; Transfer learning is performed based on the deep neural network hybrid model to adapt it to the defect detection task.

6. The method for real-time detection and location of industrial defects according to claim 1, characterized in that: The deep neural network hybrid model introduces an attention mechanism, combines the bounding box regression and key point estimation algorithms, and outputs the coordinate information of the defect area, including: By introducing the attention mechanism in the middle layer of the deep neural network hybrid model, an attention weight is generated for each position of the industrial product image; At the back end of the deep neural network hybrid model, combined with the bounding box regression or key point estimation method, the coordinate information of the defect area is output according to the recognition result of the defect area by the deep neural network hybrid model.

7. The method for real-time detection and location of industrial defects according to claim 1, characterized in that: The use of model lightweight technology and hardware acceleration technology to optimize the deep neural network hybrid model includes: Improve the reasoning and computing capabilities of deep neural network hybrid models through channel pruning, model quantization, and knowledge distillation technologies; The computational graph of the deep neural network hybrid model is optimized and mapped to the AI ​​hardware for processing.

8. A real-time industrial defect detection and positioning system, characterized in that: include: A data acquisition module, used to acquire industrial defect data images using a video acquisition device and pre-process the industrial defect data images; The data set construction module is used to annotate industrial defect data images and generate industrial defect detection data sets; A model building module for fusing convolutional neural networks with recurrent neural networks to generate a deep neural network hybrid model with spatial and temporal scales; A model training module, used for performing self-supervised pre-training and transfer learning based on the deep neural network hybrid model and the industrial defect detection dataset; The defect area recognition module is used to introduce the attention mechanism into the deep neural network hybrid model, combine the bounding box regression and key point estimation algorithm, and output the coordinate information of the defect area; Model optimization module, which is used to optimize the deep neural network hybrid model using model lightweight technology and hardware acceleration technology; The linkage detection module is used to encapsulate the deep neural network hybrid model and the corresponding algorithm into the preset industrial detection system, and perform real-time detection and positioning of industrial defects by linkage with production line equipment.

9. A real-time industrial defect detection and positioning device, characterized in that: include: A memory for storing a program for real-time detection and location of industrial defects; A processor is used to implement the steps of the method for real-time detection and positioning of industrial defects as claimed in any one of claims 1 to 7 when executing the real-time detection and positioning program of industrial defects.

10. A readable storage medium, characterized in that: The readable storage medium stores an industrial defect real-time detection and positioning program, which, when executed by a processor, implements the steps of the industrial defect real-time detection and positioning method as claimed in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Safety helmet wearing detection method in complex scene

    CN114419659A

  • Industrial defect detection method, device and equipment and readable storage medium

    CN115239638A

  • Industrial product defect detection method and device

    CN115272340A

  • Target detection and accurate positioning method based on arbitrary quadrangle regression

    CN115719414A

  • Solar cell defect detection method based on attention convolutional neural network

    CN116523820A

Cited By

  • Multi-modal production line state reasoning method, system, equipment and medium

    CN120654810A

  • Industrial equipment defect detection method based on multi-modal feature fusion and dynamic optimization

    CN121073978A

  • Industrial product quality detection method, equipment and medium

    CN121582166A

  • Industrial defect detection method based on self-supervised pre-training and feature space generation

    CN121837137A

  • Inductor surface defect detection method, system and device based on comparative learning and self-supervision

    CN121962131A