CNN-based industrial article surface damage real-time detection method and system

Through the real-time detection method based on CNN and combined with the lightweight model of Faster R-CNN and SE blocks, the problem that traditional methods are difficult to process high frame rate image data in real time is solved, and efficient and accurate detection of surface damage of industrial items is achieved. Suitable for real-time application environment.

CN119067914APending Publication Date: 2024-12-03UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202410936699.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-12
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Traditional image recognition methods are difficult to achieve real-time processing in industrial quality inspection, and the original image data has problems such as noise, blur, and occlusion, which affects the recognition performance of the CNN model and leads to inefficient surface damage detection of industrial items.

Method used

Using a real-time detection method based on CNN, the target is initially located through the pre-trained Faster R-CNN model, combined with the feature matching target tracking algorithm and the lightweight CNN recognition model of SE blocks, image preprocessing, feature extraction and real-time detection are performed. Remove noise and enhance contrast, and reduce computational resource consumption by separating convolution and inverse residual structures using depth.

Benefits of technology

It significantly improves the efficiency and accuracy of surface damage detection of industrial items, is suitable for real-time application environments, reduces storage space and computing burden, and ensures accurate detection in complex environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067914A_ABST
    Figure CN119067914A_ABST
Patent Text Reader

Abstract

The invention provides a CNN-based industrial article surface damage real-time detection method and system, and relates to the technical field of image processing, and the method comprises the steps: collecting an image of a target article in real time, and carrying out the image preprocessing; using a pre-trained Faster R-CNN model to preliminarily position the position of the target object in the image; carrying out continuous frame tracking on the detected target based on a target tracking algorithm of feature matching so as to extract a feature map of the damaged region; constructing a lightweight CNN recognition model added with an SE block, and inputting the feature map of the damage area into the lightweight CNN recognition model to obtain a real-time detection result of the target object; according to the method, a CNN recognition framework of high-frame-rate time sequence data can be processed in real time, data cleaning is effectively carried out, low-quality image data is removed or corrected, and the industrial article surface damage detection efficiency is remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a CNN-based real-time detection method and system for industrial object surface damage. Background Art

[0002] In industrial quality inspection scenarios, it is usually necessary to process high-frame-rate time series data, that is, continuous and fast image streams. Traditional image recognition methods often find it difficult to achieve real-time image processing while ensuring accuracy, resulting in low quality inspection efficiency for industrial products. In addition, when using traditional CNN models in practical applications, the original image data often has problems such as noise, blur, and occlusion, which will seriously affect the recognition performance of the CNN model, further leading to low efficiency in detecting surface damage of industrial products. Summary of the invention

[0003] The purpose of the present invention is to provide a real-time detection method for surface damage of industrial objects based on CNN, which can process the CNN recognition framework of high frame rate time series data in real time, effectively clean up data, remove or correct low-quality image data, and significantly improve the efficiency of surface damage detection of industrial objects.

[0004] The technical solution of the present invention is:

[0005] In a first aspect, the present application provides a method for real-time detection of surface damage of industrial objects based on CNN, which comprises the following steps:

[0006] S1, real-time acquisition of images of target objects and image preprocessing;

[0007] S2. Use the pre-trained Faster R-CNN model to preliminarily locate the position of the target object in the image;

[0008] S3, tracking the detected target in consecutive frames using a target tracking algorithm based on feature matching to extract a feature map of the damaged area;

[0009] S4. Construct a lightweight CNN recognition model with SE blocks added, and input the feature map of the damaged area into the lightweight CNN recognition model to obtain real-time detection results of the target object.

[0010] Furthermore, in step S1, the above-mentioned image preprocessing includes image denoising processing and image contrast enhancement processing;

[0011] The above-mentioned image denoising process uses Gaussian filtering to eliminate noise, and its calculation formula includes:

[0012]

[0013] Where G(x,y) is the Gaussian kernel function, and δ is the standard deviation of the Gaussian distribution;

[0014] The above image contrast enhancement process uses histogram equalization, and its size normalization is achieved through an interpolation algorithm, and its calculation formula is:

[0015]

[0016] In the formula, R1 and R2 are two linear interpolations in the x direction, P is the linear interpolation in the y direction, and Q 11 , Q 12 , Q 21 and Q 22 are the values ​​of the four known points.

[0017] Further, step S2 includes:

[0018] S21, input the preprocessed image into the RPN directly running on the early layer of the network to generate image candidate boxes;

[0019] S22. Divide the candidate regions by the image candidate boxes, and use the Faster R-CNN object detector to classify and locate the candidate regions to preliminarily determine the position of the target object in the image.

[0020] Furthermore, in step S3, the target tracking algorithm adopts the optical flow method, and its calculation formula includes:

[0021] I(x,y,t)=I(x+dx,y+dy,t+dt)

[0022]

[0023] Where x and y are pixel positions and t is time.

[0024] Furthermore, in step S4, the calculation process of inputting the feature map of the damaged area into the lightweight CNN recognition model to obtain the real-time detection result of the target object includes:

[0025] Perform a global average pooling operation on the input feature map of the damaged area to calculate the mean of all elements in each channel. The calculation formula is:

[0026]

[0027] In the formula, Z c is a C-dimensional channel description vector, H and W are the height and width of the feature map respectively, X c,h,w is the feature map of the input damaged area;

[0028] The compressed channel description vector Z is obtained by a two-layer fully connected network. cPerform nonlinear transformation to obtain the importance weight of each channel, and the calculation formula is:

[0029] S = ReLU(W1·Z+b1)

[0030] α c =sigmoid(W2·S+b2)

[0031] Where S is the channel weight, ReLU is the activation function, W1 is the weight matrix of the first fully connected layer, Z is the channel description vector, b1 is the bias vector of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, b2 is the bias vector of the second fully connected layer, and α c Represents the weight of each channel, and sigmoid is the normalization function;

[0032] The weight of each channel is applied to each channel of the feature map of the lesion area of ​​the original input to highlight important features. The calculation formula is:

[0033] Y c =α c ·X c,h,w

[0034] Where Y is the output feature map after adjustment by SE block.

[0035] In a second aspect, the present application provides a CNN-based real-time detection system for industrial object surface damage, comprising:

[0036] An image acquisition module is used to acquire images of target objects in real time and perform image preprocessing;

[0037] The target localization module is used to preliminarily locate the position of the target object in the image using the pre-trained Faster R-CNN model;

[0038] A feature extraction module, used for tracking the detected target in consecutive frames based on a target tracking algorithm of feature matching to extract a feature map of the damaged area;

[0039] The target detection module is used to build a lightweight CNN recognition model with SE blocks added, and input the feature map of the damaged area into the lightweight CNN recognition model to obtain real-time detection results of the target object.

[0040] In a third aspect, the present application provides an electronic device, including:

[0041] A memory for storing one or more programs;

[0042] processor;

[0043] When the one or more programs are executed by the processor, a real-time detection method for surface damage of industrial objects based on CNN as described in any one of the first aspects is implemented.

[0044] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a CNN-based real-time detection method for surface damage of industrial objects as described in any one of the first aspects above.

[0045] Compared with the prior art, the present invention has at least the following advantages or beneficial effects:

[0046] (1) The present invention uses the pre-trained Faster R-CNN model and RPN to quickly generate and optimize the candidate regions of the target image without relying on additional candidate box generation algorithms, which significantly improves the target positioning speed and accuracy, and can maintain a stable tracking effect even in scenes where the target changes dynamically;

[0047] (2) The present invention adopts an improved lightweight CNN model to make full use of the depth-separable convolution and inverted residual structure, greatly reducing the consumption of computing resources, and is particularly suitable for embedded systems or real-time application environments. At the same time, by adding SE blocks to the model, the model's learning ability for key features such as damaged areas is enhanced, and the model size is greatly compressed through pruning and quantization techniques, which not only reduces the storage space occupied, but also reduces the computational burden during real-time reasoning;

[0048] (3) The optical flow method used in the present invention can achieve precise tracking and robust extraction of damaged areas between consecutive frames without being affected by changes in the target state, ensuring that the damage information of the target object can be accurately captured even in complex environments. This not only ensures data security, but also effectively saves network resources and computing resources, ultimately achieving real-time, efficient and accurate industrial object detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0050] Figure 1 A step diagram of a real-time detection method for surface damage of industrial objects based on CNN of the present invention;

[0051] Figure 2 The figure is a schematic structural block diagram of an electronic device.

[0052] Icon: 101, memory; 102, processor; 103, communication interface. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0054] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0055] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0056] It should be noted that, in this article, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprises..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0057] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0058] Example 1

[0059] See also Figure 1 , Figure 1 Shown is a step diagram of a CNN-based real-time detection method for surface damage of industrial objects provided in an embodiment of the present application.

[0060] In a first aspect, the present application provides a method for real-time detection of surface damage of industrial objects based on CNN, which comprises the following steps:

[0061] S1, real-time acquisition of images of target objects and image preprocessing;

[0062] S2. Use the pre-trained Faster R-CNN model to preliminarily locate the position of the target object in the image;

[0063] S3, tracking the detected target in consecutive frames using a target tracking algorithm based on feature matching to extract a feature map of the damaged area;

[0064] S4. Construct a lightweight CNN recognition model with SE blocks added, and input the feature map of the damaged area into the lightweight CNN recognition model to obtain real-time detection results of the target object.

[0065] As a preferred implementation, in step S1, the image preprocessing includes image denoising and image contrast enhancement;

[0066] Among them, the image denoising process uses Gaussian filtering to eliminate noise, and its calculation formula includes:

[0067]

[0068] Where G(x,y) is the Gaussian kernel function, and δ is the standard deviation of the Gaussian distribution;

[0069] Image contrast enhancement processing uses histogram equalization, and its size normalization is achieved through interpolation algorithm. The calculation formula is:

[0070]

[0071] In the formula, R1 and R2 are two linear interpolations in the x direction, P is the linear interpolation in the y direction, and Q 11 , Q 12 , Q 21 and Q 22 are the values ​​of the four known points.

[0072] As a preferred implementation, step S2 includes:

[0073] S21, input the preprocessed image into the RPN directly running on the early layer of the network to generate image candidate boxes;

[0074] S22. Divide the candidate regions by the image candidate boxes, and use the Faster R-CNN object detector to classify and locate the candidate regions to preliminarily determine the position of the target object in the image.

[0075] Specifically, Faster R-CNN mainly consists of two modules:

[0076] The first module is RPN, which runs directly on top of the early layers of the network and takes the entire image as input without the need for additional external algorithms to generate candidate boxes. It generates multiple anchor boxes of different scales and ratios, and classifies and regresses these anchors to predict whether each anchor contains an object and how to adjust the anchor to more accurately fit the bounding box of the potential target object. Its output is multiple sets of candidate box coordinates and their corresponding scores, which reflect the possibility of containing the target object in the box. This process greatly improves the speed and accuracy of generating candidate regions; the present invention refines the design of the RPN to be used, adopts multi-scale anchor box prediction to adapt to targets of different sizes, and adjusts the anchor ratio to better match various target shapes that may appear during tracking. Spatial pyramid pooling is introduced to enhance the sensitivity of RPN to target position and contextual information;

[0077] The second module is the Fast R-CNN object detector based on the candidate box, which is used to classify and locate the candidate area. For each ROI generated by RPN, Fast R-CNN will extract features, that is, map the image features in the ROI area to a fixed-dimensional space, and then perform classification and bounding box regression through the fully connected layer. The classification part is used to determine the category of the object in the candidate box, while the bounding box regression further optimizes the position and size of the candidate box to make it more closely fit the actual contour of the target object. The object detector outputs the probability of each ROI area being identified as an object of each category, as well as the corresponding corrected bounding box coordinates.

[0078] As a preferred implementation, in step S3, the target tracking algorithm adopts an optical flow method, and its calculation formula includes:

[0079] According to the principle of constant brightness:

[0080] I(x,y,t)=I(x+dx,y+dy,t+dt)

[0081] After simplification, the optical flow equation is obtained by Taylor expansion and ignoring high-order terms:

[0082]

[0083] Where x and y are pixel positions and t is time.

[0084] It should be noted that step S3 is based on step S2, and uses a target tracking algorithm based on feature matching to track the detected target in consecutive frames to ensure that the target can still be accurately tracked when it moves, deforms or is blocked. During the tracking process, the ROI image of each target object is extracted. The ROI image is a local area containing key information of the target object. It removes background noise and irrelevant information, providing more accurate and effective data for subsequent damage detection. Then the feature matching algorithm uses the optical flow method to calculate and predict the position of the target in the next frame by comparing the feature changes of the target object between adjacent frames, and can maintain stable tracking even if the target state changes. Since the target object may move, deform or be partially blocked in the video stream, in order to ensure that the target can still be accurately tracked in these cases, the optical flow method can estimate the movement and deformation of the target object between consecutive frames.

[0085] Since the system of equations is underdetermined, further constraints are required. This embodiment uses the Lucas-Kanade algorithm to solve the optical flow problem. The main idea is that under the assumption of small displacement, the movement of pixels in the local area is approximately consistent. The algorithm minimizes a cost function through iterative optimization. This cost function usually includes two components: data term: reflects the difference in brightness invariance between the corresponding pixels in the current frame and the previous frame, that is, the residual in the optical flow equation. By constructing a function about the optical flow vector (dx, dy), the function measures whether the pixel brightness difference is minimized under a given motion vector; regularization term: in order to overcome the underdetermination of the optical flow equation, the smoothness constraint of the optical flow field is introduced. This means that the motion vectors of adjacent pixels should be as close as possible, because the movement of objects in natural scenes is often continuous and smooth.

[0086] According to the bounding box of the target object detected by Faster R-CNN, the ROI image of the region of interest is extracted. The region only contains the target object without background noise and irrelevant information. Then, the ROI image is subjected to feature extraction. For the feature extraction part, HOG is used (HOG feature descriptor is a feature of the local gradient directional histogram of a statistical image. It is particularly sensitive to the directional information of shapes and edges, and is particularly suitable for target detection and recognition tasks). When the HOG algorithm is applied to the ROI image, the image is divided into multiple small cell units, the directional histogram of the pixel gradient in each unit is calculated, and then the adjacent units are connected to form a larger block. The histogram information in the block is combined to form a compact and rotationally invariant feature vector. When the target object moves in continuous frames, the HOG features are extracted from the ROI images of the previous and next frames respectively, and the best matching feature point pair is found by using the nearest neighbor search to obtain a set of vectors describing the characteristics of the target object. Then, in continuous frames, the optimal matching pair is found by comparing the feature descriptors of the previous and next frames, and the motion vector (dx, dy) of the object is estimated. If the target is deformed, it may be necessary to combine more feature points and geometric transformation models (such as affine transformation, perspective transformation, etc.) for tracking. Finally, the optical flow method is used to predict the position of the target object in the next frame, which can maintain a relatively stable tracking effect even when the target state changes. This real-time tracking helps to continuously monitor the target object and promptly discover and extract possible damaged areas.

[0087] As a preferred implementation, in step S4, the calculation process of inputting the feature map of the damaged area into the lightweight CNN recognition model to obtain the real-time detection result of the target object includes:

[0088] Perform a global average pooling operation on the input feature map of the damaged area to calculate the mean of all elements in each channel. The calculation formula is:

[0089]

[0090] In the formula, Z c is a C-dimensional channel description vector, H and W are the height and width of the feature map respectively, X c,h,w is the feature map of the input damaged area;

[0091] The compressed channel description vector Z is obtained by a two-layer fully connected network. c Perform nonlinear transformation to obtain the importance weight of each channel, and the calculation formula is:

[0092] S = ReLU(W1·Z+b1)

[0093] α c =sigmoid(W2·S+b2)

[0094] Where S is the channel weight, ReLU is the activation function, W1 is the weight matrix of the first fully connected layer, Z is the channel description vector, b1 is the bias vector of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, b2 is the bias vector of the second fully connected layer, and α c Represents the weight of each channel, and sigmoid is the normalization function;

[0095] The weight of each channel is applied to each channel of the feature map of the lesion area of ​​the original input to highlight important features. The calculation formula is:

[0096] Y c =α c ·X c,h,w

[0097] Where Y is the output feature map after adjustment by SE block.

[0098] It should be noted that this step is the detailed mathematical principle process of the SE block. Through the SE block, the network can dynamically adjust the response of each channel, emphasize the features that are important to the recognition task, and weaken irrelevant or redundant features, thereby improving the performance of the model. This embodiment makes improvements to the lightweight model MobileNet. Its core innovation is to use deep separable convolution to replace the traditional convolution layer. The deep separable convolution is divided into two steps: first, perform a deep convolution (DepthwiseConvolution), and each channel performs a convolution operation independently, which greatly reduces the amount of calculation; then perform pointwise convolution (PointwiseConvolution) to mix information from different channels. The conventional N×N convolution kernel is split into a sequence of 1×N and N×1 convolution kernels to reduce computing resource consumption. Drawing on designs similar to SqueezeNet, the bottleneck layer structure is used to reduce the complexity of the model while maintaining good performance. The inverted residual structure is introduced in the MobileNet v2 version. The structure contains shortcut connections, but is improved on the basis of the traditional residual structure, and efficient feature transfer is achieved through expansion and contraction operations. Then, a SE block (Squeeze-and-Excitation) is added to the original model to improve the model's attention to key areas; in order to reduce model overhead, redundant parameters are removed through pruning, and then the remaining model parameters are quantized, which can greatly reduce the model size and computational cost while ensuring model performance, making MobileNet more suitable for real-time image recognition tasks.

[0099] Example 2

[0100] In a second aspect, the present application provides a CNN-based real-time detection system for industrial object surface damage, comprising:

[0101] An image acquisition module is used to acquire images of target objects in real time and perform image preprocessing;

[0102] The target localization module is used to preliminarily locate the position of the target object in the image using the pre-trained Faster R-CNN model;

[0103] A feature extraction module, used for tracking the detected target in consecutive frames based on a target tracking algorithm of feature matching to extract a feature map of the damaged area;

[0104] The target detection module is used to build a lightweight CNN recognition model with SE blocks added, and input the feature map of the damaged area into the lightweight CNN recognition model to obtain real-time detection results of the target object.

[0105] The embodiment of the present application provides a CNN-based real-time detection system for surface damage of industrial objects. The image acquisition module is used to acquire images of target objects in real time and perform efficient image preprocessing. Then, the target positioning module and the feature extraction module are used to perform advanced target detection and tracking technology. Finally, the target detection module is designed with a lightweight CNN model optimized for real-time application scenarios, which not only ensures data security but also effectively saves network resources and computing resources, ultimately achieving the purpose of real-time, efficient and accurate industrial object detection and analysis.

[0106] Example 3

[0107] See also Figure 2 , Figure 2 A schematic structural block diagram of an electronic device provided in an embodiment of the present application.

[0108] An electronic device includes a memory 101, a processor 102 and a communication interface 103, wherein the memory 101, the processor 102 and the communication interface 103 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, and the processor 102 executes various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used to communicate signaling or data with other node devices.

[0109] Among them, the memory 101 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc.

[0110] The processor 102 may be an integrated circuit chip with signal processing capability. The processor 102 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0111] It is understood that the structure shown in the figure is only for illustration, and a method and system for real-time detection of surface damage of industrial objects based on CNN may also include more or fewer components than those shown in the figure, or have a different configuration than those shown in the figure. Each component shown in the figure may be implemented by hardware, software or a combination thereof.

[0112] In the embodiments provided in the present application, it should be understood that the disclosed method can also be implemented in other ways. The embodiments described above are merely schematic, for example, the flowchart or block diagram in the accompanying drawings shows the possible implementation architecture, functions and operations of the method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of the boxes in the block diagram and / or the flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0114] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0115] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0116] It will be apparent to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A real-time detection method for surface damage of industrial objects based on CNN, characterized in that: The following steps are involved: S1, real-time acquisition of images of target objects and image preprocessing; S2. Use the pre-trained Faster R-CNN model to preliminarily locate the position of the target object in the image; S3, tracking the detected target in consecutive frames using a target tracking algorithm based on feature matching to extract a feature map of the damaged area; S4. Construct a lightweight CNN recognition model with SE blocks added, and input the feature map of the damaged area into the lightweight CNN recognition model to obtain real-time detection results of the target object.

2. A method for real-time detection of surface damage of industrial objects based on CNN as claimed in claim 1, characterized in that: In step S1, the image preprocessing includes image denoising and image contrast enhancement; The image denoising process uses Gaussian filtering to eliminate noise, and its calculation formula includes: Where G(x,y) is the Gaussian kernel function, and δ is the standard deviation of the Gaussian distribution; The image contrast enhancement process adopts histogram equalization, and its size normalization is achieved through an interpolation algorithm, and its calculation formula is: In the formula, R1 and R2 are two linear interpolations in the x direction, P is the linear interpolation in the y direction, and Q 11 , Q 12 , Q 21 and Q 22 are the values ​​of the four known points.

3. A method for real-time detection of surface damage of industrial objects based on CNN as claimed in claim 1, characterized in that: Step S2 includes: S21, input the preprocessed image into the RPN directly running on the early layer of the network to generate image candidate boxes; S22. Divide the candidate regions by the image candidate boxes, and use the Faster R-CNN object detector to classify and locate the candidate regions to preliminarily determine the position of the target object in the image.

4. A method for real-time detection of surface damage of industrial objects based on CNN as claimed in claim 1, characterized in that: In step S3, the target tracking algorithm adopts the optical flow method, and its calculation formula includes: I(x,y,t)=I(x+dx,y+dy,t+dt) Where x and y are pixel positions and t is time.

5. The real-time detection method for surface damage of industrial objects based on CNN as claimed in claim 1, characterized in that: In step S4, the calculation process of inputting the feature map of the damaged area into the lightweight CNN recognition model to obtain the real-time detection result of the target object includes: Perform a global average pooling operation on the input feature map of the damaged area to calculate the mean of all elements in each channel. The calculation formula is: In the formula, Z c is a C-dimensional channel description vector, H and W are the height and width of the feature map respectively, X c,h,w is the feature map of the input damaged area; The compressed channel description vector Z is obtained by a two-layer fully connected network. c Perform nonlinear transformation to obtain the importance weight of each channel, and the calculation formula is: S = ReLU(W1·Z+b1) α c =sigmoid(W2·S+b2) Where S is the channel weight, ReLU is the activation function, W1 is the weight matrix of the first fully connected layer, Z is the channel description vector, b1 is the bias vector of the first fully connected layer, W2 is the weight matrix of the second fully connected layer, b2 is the bias vector of the second fully connected layer, and α c Represents the weight of each channel, and sigmoid is the normalization function; The weight of each channel is applied to each channel of the feature map of the lesion area of ​​the original input to highlight important features. The calculation formula is: Y c =a c ·X c,h,w Where Y is the output feature map after adjustment by SE block.

6. A real-time detection system for industrial object surface damage based on CNN, characterized in that: include: An image acquisition module is used to acquire images of target objects in real time and perform image preprocessing; The target localization module is used to preliminarily locate the position of the target object in the image using the pre-trained Faster R-CNN model; A feature extraction module, used for tracking the detected target in consecutive frames based on a target tracking algorithm of feature matching to extract a feature map of the damaged area; The target detection module is used to build a lightweight CNN recognition model with SE blocks added, and input the feature map of the damaged area into the lightweight CNN recognition model to obtain real-time detection results of the target object.

7. An electronic device, characterized in that: include: A memory for storing one or more programs; processor; When the one or more programs are executed by the processor, a real-time detection method for surface damage of industrial objects based on CNN as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a real-time detection method for surface damage of industrial objects based on CNN as described in any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Metal plate strip surface defect detecting method and device based on Faster R-CNN network

    CN106952250A

  • Infrared-video-based target detecting and tracking method and device

    CN107784291A

  • Real-time smoke detection method based on deep learning and optical flow method

    CN110991245A

  • Multi-target tracking method and system based on graph matching

    CN111862156A

  • Tower crane crack image detection method

    CN112241950A