A virtual edge detection method for coated ceramic area based on deep learning

Through the improved YOLOv5 network and deep learning technology, combined with local contrast enhancement and non-maximum suppression algorithm, the problem of accurate positioning of virtual edge detection in coated ceramic areas is solved, and high-precision and efficient virtual edge detection is achieved, reducing artificial Operating cost.

CN118674723BActive Publication Date: 2024-11-15NANJING HUASHI INTELLIGENT TECH CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202411163451.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-11-15
Estimated Expiration
2044-08-23

AI Technical Summary

Technical Problem

The prior art is difficult to accurately locate and detect the virtual edges of coated ceramic areas. The size of the virtual edge detected by traditional algorithms fluctuates greatly, the manual operation cost is high, and there is a risk of false detection. The conventional object detection algorithm is not ideal for the virtual edge detection.

Method used

Using the virtual edge detection method of coated ceramic areas based on deep learning, image data is collected through industrial cameras for pre-processing, and virtual edge detection is performed using the improved YOLOv5 network, introducing attention mechanism, adding a small object detection layer, adjusting the upsampling method and The loss function, combined with local contrast enhancement and non-maximum suppression algorithm, realizes accurate detection of virtual edges.

Benefits of technology

It improves the accuracy and speed of virtual edge detection, avoids human operation errors, ensures the accuracy and reliability of virtual edge detection results, and reduces operating costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118674723B_ABST
    Figure CN118674723B_ABST
Patent Text Reader

Abstract

The present invention relates to a method for detecting virtual edges of coated ceramic areas based on deep learning, which belongs to the technical field of target detection. Coating image data is collected, and the pre-processed coating image is input into a ceramic area detection model based on lightweight network YOLOv5-Lite training to obtain the ceramic area position and type; a coarsely positioned virtual edge ROI area is constructed, and the ROI area is mapped back to the original image to obtain a virtual edge image to be detected, and local contrast enhancement is performed on the virtual edge image to be detected; an improved YOLOv5 network is established as a virtual edge detection model, and the virtual edge image to be detected is input into a trained virtual edge detection model, and the model outputs a virtual edge detection result; the virtual edge detection result is analyzed to obtain the position and size information of the virtual edge in the original coating image coordinate system. The present invention realizes high-precision real-time virtual edge detection in a complex coating production environment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a method for detecting virtual edges of a coated ceramic area based on deep learning, and belongs to the technical field of target detection. Background Art

[0002] In the lithium battery coating production process, in order to prevent the positive and negative electrode materials from directly contacting each other and causing a short circuit, a layer of insulating ceramic is usually applied at the junction of the positive electrode tab and the carbon film. However, during the coating and baking process, since both the carbon film and the ceramic coating are water-based slurries, the two are very easy to dissolve into each other, forming the so-called "virtual edge", which leads to problems such as difficulty in grasping the edge and large misalignment of the A and B surfaces in subsequent processes.

[0003] At present, the virtual edge detection of coated ceramic areas mainly relies on traditional algorithms, that is, detection is performed within a manually selected grabbing frame using a caliper tool. This method has the following disadvantages: 1. The grayscale difference on both sides of the virtual edge is relatively small, and the boundary is relatively blurred. It is difficult for traditional algorithms to accurately locate the virtual edge, and the size of the detected virtual edge fluctuates greatly; 2. When the edge of the ceramic area is uneven, or when the coated product is changed, the manually selected grabbing frame may not cover the virtual edge position, and various grabbing parameters need to be adjusted frequently, which has high operating costs and the risk of edge grabbing failure due to human operating errors.

[0004] With the development and popularization of deep learning technology, the use of deep learning algorithms for virtual edge detection has become a new solution. However, virtual edge detection is different from the detection of conventional objects. Its pixel ratio relative to the entire coated image is very small, and the aspect ratio is very large. A virtual edge usually spans the entire coated image. Conventional target detection algorithms are not ideal for virtual edge detection. Moreover, the coating size is too large. In order to ensure the detection speed, it is usually necessary to scale the original large-size image, which will lead to a decrease in detection accuracy. It can be seen that how to accurately detect the virtual edges of coated ceramic areas is still a social and technical problem facing the industry. Summary of the invention

[0005] In order to solve the above problems, the present invention discloses a method for detecting virtual edges of coated ceramic areas based on deep learning, and its specific technical solution is as follows:

[0006] A method for detecting virtual edges of coated ceramic areas based on deep learning, comprising the following steps:

[0007] Step S1, collecting coating image data through an industrial camera, and preprocessing the collected data, the preprocessing includes: grayscale, Gaussian blur and image scaling;

[0008] Step S2, inputting the pre-processed coating image into the ceramic area detection model to obtain the position and type of the ceramic area;

[0009] Step S3, constructing a roughly located virtual edge ROI region according to the position and type of the ceramic area, mapping the virtual edge ROI region back to the original image to obtain a virtual edge image to be detected, and performing local contrast enhancement on the virtual edge image to be detected;

[0010] Step S4, improving the original YOLOv5 network by introducing an attention mechanism, adding a small target detection layer, changing the upsampling method and the loss function, obtaining an improved YOLOv5 network, training the improved YOLOv5 network using a virtual edge detection dataset, obtaining a virtual edge detection model, inputting the virtual edge image to be detected into the virtual edge detection model, and the virtual edge detection model outputs a virtual edge detection result;

[0011] Step S5, analyzing the virtual edge detection result to obtain the position and size information of the virtual edge in the original coating image coordinate system.

[0012] Furthermore, in step S2, a lightweight target detection network YOLOv5-Lite is used as a ceramic area detection model, and a non-maximum suppression algorithm is used to remove redundant information in the model output to obtain the position and type of each ceramic area in the coating image;

[0013] The types of the ceramic region include a carbon film left ceramic region and a carbon film right ceramic region.

[0014] Furthermore, in step S3, the positional relationship between the ceramic area and the carbon film area is determined according to the position and type of the ceramic area, and the edges of the boundaries between the adjacent ceramic area and the carbon film area are used as references to expand to the left and right sides respectively, and the ROI area where the virtual edge is located is preliminarily located. The ROI area is mapped back to the original image for cropping. Since the virtual edge size only requires the calculation of width information but not height, the cropped image is scaled in the height direction while keeping the width unchanged to obtain the virtual edge image to be detected, ensuring that the virtual edge detection size has pixel-level accuracy while greatly improving the detection speed, and then the local contrast enhancement is performed on the virtual edge image to be detected as the input of the virtual edge detection model.

[0015] Furthermore, the specific implementation steps of the local contrast enhancement are:

[0016] The virtual edge image to be detected is divided into several small blocks, and each small block is subjected to histogram equalization. The high-frequency grayscale value is limited by the contrast threshold, and the pixels exceeding the limit are evenly distributed to other parts of the histogram to limit the contrast of each small block. The calculation formula is:

[0017]

[0018] In the formula, S ij (k) indicates that at gray level k, the small block T ijThe gray value after processing; L is the total number of gray levels; H ij (m) is the small piece T ij The number of pixels with a gray value of m; C is the contrast threshold, which is used to limit the high-frequency part of the histogram; For small block T ij The sum of the remaining histogram counts after subtracting the contrast threshold C from the histogram counts of all gray levels l in |T ij |For small pieces of T ij The total number of pixels, i.e. the area of ​​the small block;

[0019] Bilinear interpolation is used to smooth the boundaries between small blocks, eliminate the block effect, and smoothly merge all small blocks to form a complete contrast-enhanced virtual edge image to be detected.

[0020] Further, in step S4, the virtual edge detection model is: annotating the virtual edge in the virtual edge image to be detected so that the left and right sides of the calibration frame coincide with the left and right sides of the virtual edge to obtain a virtual edge data set, and inputting the virtual edge data set into the improved YOLOv5 model for training to obtain a virtual edge detection model;

[0021] The improved YOLOv5 network includes the feature extraction network Backbone, the feature fusion network Neck and the detection head Head. The specific improvements to the YOLOv5 network include:

[0022] (1) Introduce the coordinate attention mechanism CA into the C3 module of the feature extraction network Backbone network in the YOLOv5 network. The specific operation steps are as follows:

[0023] The feature map obtained after the 3×3 convolution in the C3 module is globally averaged pooled using pooling kernels of size C'×1×W and C'×H×1 along the height and width directions, respectively, and the position information is encoded in a pair of direction-aware feature vectors, which is calculated as follows:

[0024]

[0025]

[0026] Among them, C', H, W represent the number of channels, height and width of the input feature map respectively; X∈R C’×H×W represents the input feature map; X h ∈R C'×1×W , X w ∈R C'×H×1 Represent the feature vectors after global average pooling in the horizontal and vertical directions respectively;

[0027] After swapping the dimensions of one of the feature vectors, it is concatenated with the other feature vector and input into a transformation function shared by both. The calculation formula is:

[0028] f=δ(F1(cat[X h ,X′ w ]))

[0029] Among them, δ is a nonlinear activation function; F1 is a 1×1 convolution transformation function; X' w Represents X after the width and height dimensions are swapped w ; cat[X h ,X' w ] indicates that X h and X' w Concatenate the corresponding dimensions; f∈R C' / r×(H+W) is the intermediate feature vector that integrates the position information in both height and width directions, r is the maximum downsampling ratio, which is used to control the module size;

[0030] Split f into two independent vectors f along the spatial dimension h ∈R C' / r×H and f w ∈R C' / r×W Input into two 1×1 convolution transformation functions F respectively h and F w In, we get:

[0031] g h =σ(F h (f h ))

[0032] g w =σ(F w (f w ))

[0033] Among them, f h and f w are the feature maps split along the height and width directions respectively; F h and F w are 1×1 convolution transformation matrices along the height and width directions respectively; σ is the sigmoid activation function; g h and g w They are the attention weight maps obtained after convolution transformation and activation function in the height and width directions, respectively, which are used to adjust the weights of the input feature maps in the corresponding directions;

[0034] The obtained g h and g wThe feature vectors that are expanded along the width and height directions to have the same size as the input feature map X are the CA weights. Multiplying the CA weights by the corresponding positions of the original input feature map will give an output with stronger feature expression capabilities:

[0035]

[0036] Where, c represents the cth channel; i and j represent the horizontal and vertical coordinate positions respectively;

[0037] (2) Add a small target detection layer and an anchor frame for the small target detection layer. The specific steps are as follows:

[0038] In YOLOv5, the feature fusion network Neck is upsampled once more based on the 2nd upsampling. The obtained feature map is spliced ​​with the second layer of the feature extraction network Backbone to obtain a large-resolution feature map for small target detection. Then, the normal downsampling operation is continued. Finally, the 21st, 24th, 27th, and 30th layer feature maps of the YOLOv5 network are used as the input of the detection head Head.

[0039] Based on the virtual edge detection dataset, 12 anchor boxes are regenerated using the K-means clustering algorithm;

[0040] (3) Use lightweight dynamic upsampling DySample to replace the bilinear interpolation upsampling in the YOLOv5 network;

[0041] (4) Adjust the loss function. The specific steps are as follows:

[0042] CIoULoss (complete intersection-over-union loss) is changed to EIoULoss (efficient intersection-over-union loss), and the border regression loss weight of the small target detection layer is appropriately increased. The calculation formula is as follows:

[0043]

[0044] Where IOU is the intersection-over-union ratio between the predicted box and the real box; ρ is the Euclidean distance between two points; b, w, h are the coordinates of the center point, width, and height of the predicted box; b gt 、w gt 、h gt w is the center point coordinates, width and height of the real frame; c 、h c is the width and height of the minimum outer rectangle of the predicted box and the real box; Loss EIOU is the EIoU loss value, which measures the error between the predicted bounding box and the true bounding box;

[0045] Change the confidence and classification loss from CrossEntropyLoss (cross entropy loss) to FocalLoss (focal loss), and the calculation formula is as follows:

[0046] Loss Focal (p t )=-α t (1-p t ) γ log(p t )

[0047] In the formula, p t Represents the model's predicted probability for the correct category; α t is a balancing factor used to balance the ratio of positive and negative samples; γ is an adjustment factor used to adjust the loss weight of easy-to-classify samples; Loss Focal (p t ) is the Focal Loss value, which measures the classification error of the model when dealing with unbalanced samples.

[0048] Furthermore, a confidence threshold is set to filter the virtual edge detection results, and at most one detection result with the highest confidence and higher than the threshold is retained in each virtual edge image to be detected;

[0049] The horizontal coordinates of the left and right sides of the target frame in the detection result can represent the positions of the left and right edges of the virtual edge, and the absolute value of the difference between the two horizontal coordinates is the width of the virtual edge;

[0050] Combined with the position information of the virtual edge image to be detected in the original image, the virtual edge coordinates are mapped to the original image to obtain the position and size information of the virtual edge in the original coated image.

[0051] The beneficial effects of the present invention are:

[0052] (1) The present invention utilizes two cascaded target detection models as a virtual edge detection tool for the coated ceramic area. Since the virtual edge image to be detected is constructed based on the ceramic area output by the first model, and the model output result is updated in real time, the virtual edge detection area will always cover the location of the virtual edge, and there is no problem of the frame deviating from the virtual edge area, thereby avoiding missed detection and false detection of virtual edges caused by human operational errors.

[0053] (2) Since the coated image is too large and the virtual edge accounts for a very small proportion of pixels relative to the entire coated image, if the original image is directly subjected to virtual edge detection, the detection time will be too long and it will be difficult to accurately locate the virtual edge. Therefore, the present invention adopts a hierarchical detection strategy. First, a lightweight target detection model is used to detect ceramic areas that are easy to detect and have a large contrast in the scaled coated image. The virtual edge area is roughly located based on the ceramic area information, and a smaller virtual edge image to be detected is constructed. Subsequently, an improved YOLOv5 model is used to accurately locate the virtual edge in the unscaled virtual edge image to be detected. This method has high detection accuracy and low time consumption.

[0054] (3) Virtual edges have the characteristics of being slender, having a high aspect ratio, and having blurred edges, and conventional target detection algorithms have unsatisfactory detection effects on virtual edges. Therefore, the present invention improves YOLOv5 to improve the accuracy of virtual edge detection, specifically: introducing the CA attention mechanism, combining channel information and direction-related position information, so that the model can better capture these directional features; adding a small target detection layer, so that the model can maintain efficient detection capabilities when facing small-scale virtual edges; as a small target, the virtual edge itself contains less feature information, and DySample upsampling is used to reduce feature loss during the upsampling process, which is a parameter that needs to be supplemented in the dynamic learning model; the EIoU loss takes into account the aspect ratio and center point distance of the target box, and replaces the positioning loss from CIoU loss to EIoU loss so that the target box can be more closely aligned with the actual shape of the slender target, thereby improving the accuracy of virtual edge positioning; using Focal Loss to replace Cross Entropy Loss makes the model pay more attention to those small, blurred or difficult-to-distinguish virtual edges. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a flow chart of the present invention;

[0056] Figure 2 It is a diagram of the improved YOLOv5 network structure of the present invention;

[0057] Figure 3 It is a schematic diagram of various areas in a common coating image;

[0058] Figure 4 It is a virtual edge detection effect diagram of the present invention. DETAILED DESCRIPTION

[0059] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0060] like Figure 1 As shown, this embodiment discloses a method for detecting virtual edges of coated ceramic areas based on deep learning, comprising the following steps:

[0061] S1. Collect coating image data through industrial cameras and pre-process the collected data, including grayscale, Gaussian blur and image scaling;

[0062] S2. Input the preprocessed coating image into the ceramic area detection model trained based on the lightweight network YOLOv5-Lite to obtain the location and type of the ceramic area;

[0063] S3, constructing a roughly located virtual edge ROI region according to the ceramic area information, mapping the ROI region back to the original image, obtaining a virtual edge image to be detected, and performing local contrast enhancement on the virtual edge image to be detected;

[0064] S4. Improve the original YOLOv5 network by introducing the attention mechanism, adding a small target detection layer, changing the upsampling method and loss function, etc. Input the virtual edge dataset into the improved YOLOv5 model for training to obtain a virtual edge detection model, input the virtual edge image to be detected into the trained virtual edge detection model, and the model outputs the virtual edge detection result;

[0065] The specific training process is as follows: the virtual edges in the virtual edge image to be detected are marked so that the left and right sides of the calibration box coincide with the left and right sides of the virtual edge to obtain a virtual edge data set, and the virtual edge data set is input into the improved YOLOv5 model for training to obtain a virtual edge detection model.

[0066] S5. Analyze the virtual edge detection result to obtain the position and size information of the virtual edge in the original coating image coordinate system.

[0067] In an optional embodiment, the size of the collected coating image data is 16384×5000 (width×height), and is scaled to 4096×320 after pre-processing such as grayscale and Gaussian filtering.

[0068] In an optional embodiment, the ceramic area detection model in step S2: the annotation set of the ceramic area is taken as input and input into the lightweight object detection network YOLOv5-Lite for training to obtain the ceramic area detection model.

[0069] In an optional embodiment, the preprocessed coating image is input into a trained ceramic area detection model, and a non-maximum suppression algorithm is used to remove redundant information in the model output to obtain the position and type of each ceramic area in the coating image;

[0070] The ceramic area types include a ceramic area on the left side of the carbon film and a ceramic area on the right side of the carbon film.

[0071] In an optional embodiment, since the virtual edge is located at the junction of the ceramic area and the carbon film area, both sides of the carbon film area are ceramic areas. Figure 3When the ceramic area is on the left side of the carbon film, the right edge of the ceramic area is used as the reference to expand 160 pixels on both sides. When the ceramic area is on the right side of the carbon film, the left edge of the ceramic area is used as the reference to expand 160 pixels on both sides. Then, a ROI area of ​​320×5000 containing a virtual edge is obtained. The ROI area is mapped back to the original image for cropping. Since the virtual edge size only requires the calculation of width information but not height, the height of the cropped image is scaled to 160 while keeping the width of 320 unchanged. The virtual edge image to be detected is 320×160 in size, ensuring that the virtual edge detection size has pixel-level accuracy while greatly improving the detection speed. Local contrast enhancement is then performed on the virtual edge image to be detected as the input of the virtual edge detection model.

[0072] The specific implementation steps for the local contrast enhancement are:

[0073] The virtual edge image to be detected is divided into several small blocks, and each small block is subjected to histogram equalization separately. The contrast of the small blocks is limited by cutting the high-frequency grayscale values ​​and distributing them evenly to other parts of the histogram. The calculation formula is:

[0074] The virtual edge image to be detected is divided into several small blocks, and each small block is subjected to histogram equalization. The high-frequency grayscale value is limited by the contrast threshold, and the pixels exceeding the limit are evenly distributed to other parts of the histogram to limit the contrast of each small block. When performing histogram equalization on each small block, the number of pixels of certain grayscale values ​​may exceed the preset contrast limit threshold C. In order to prevent the number of pixels of certain grayscale levels (0-255) from being too large (causing the image to be over-enhanced), the part exceeding the threshold will be evenly distributed to other grayscale levels of the histogram. The grayscale levels that do not exceed the threshold C will receive and distribute these excess pixels. The calculation formula is:

[0075]

[0076] In the formula, S ij (k) is a small piece T ij The gray value after processing, for gray level k; L is the total number of gray levels; H ij (m) is the small piece T ij The number of pixels with a gray value of m; C is the contrast limit parameter, which is used to limit the high-frequency part of the histogram; For small block T ij The sum of the remaining histogram counts after subtracting the contrast limit parameter C from the histogram counts of all gray levels l in T ij For small pieces of T ij The total number of pixels, i.e. the area of ​​the small block;

[0077] Bilinear interpolation is used to smooth the boundaries between small blocks, eliminate the block effect, and smoothly merge these small blocks to form a complete contrast-enhanced virtual edge image to be detected.

[0078] See also Figure 2 , the YOLOv5 model includes a feature extraction network (Backbone), a feature fusion network (Neck) and a detection head (Head). Figure 2 The meanings of the letter combinations are:

[0079] CBS: a series of ordinary convolutional layers + BN normalization layers + Silu activation function layers.

[0080] Concat: concatenation layer, which means concatenating the processed feature maps in the channel dimension.

[0081] Bottleneck: Bottleneck structure, which is two CBSs connected in series when used in Backbone and has a residual structure when used in Neck.

[0082] CABottleneck: Introduces the CA (CoordAtt) attention mechanism into the residual structure main branch of Bottleneck.

[0083] C3_X: After concatenating the feature map obtained by CBS and Bottleneck in series with the feature map obtained by another CBS, CBS is performed again. X represents the number of times Bottleneck is executed in the module.

[0084] CAC3_X: Replace Bottleneck in the C3_X module with CABottleneck, where X represents the number of times CABottleneck is executed in the module.

[0085] DySample: Dynamic upsampling module.

[0086] SPPF: Spatial Pyramid Pooling.

[0087] Specific improvements to YOLOv5 include:

[0088] (1) Introduce the coordinate attention mechanism CA (CoordAtt) into the C3 module of the Backbone network in YOLOv5. The specific operation steps are as follows:

[0089] The feature map obtained after the 3×3 convolution in the C3 module is globally averaged pooled using pooling kernels of size C'×1×W and C'×H×1 along the height and width directions, respectively, and the position information is encoded in a pair of direction-aware feature vectors, which is calculated as follows:

[0090]

[0091]

[0092] Among them, C', H, W represent the number of channels, height and width of the input feature map respectively; X∈R C'×H×W represents the input feature map; X h ∈R C'×1×W , X w ∈R C'×H×1 Represent the feature vectors after global average pooling in the horizontal and vertical directions respectively.

[0093] After swapping the dimensions of one of the feature vectors, it is concatenated with the other feature vector and input into a transformation function shared by both. The calculation formula is:

[0094] f=δ(F1(cat[X h ,X' w ]))

[0095] Among them, δ is a nonlinear activation function; F1 is a 1×1 convolution transformation function; X' w Represents X after the width and height dimensions are swapped w ; cat[X h ,X' w ] indicates that X h and X' w Concatenate the corresponding dimensions; f∈R C' / r×(H+W) It is the intermediate feature vector that integrates the position information in both height and width directions, and r is the maximum downsampling ratio, which is used to control the module size.

[0096] Split f into two independent vectors f along the spatial dimension h ∈R C' / r×H and f w ∈R C' / r×W Input into two 1×1 convolution transformation functions F respectively h and F w In, we get:

[0097] g h =σ(F h (f h ))

[0098] g w =σ(F w (f w ))

[0099] Among them, f h and f w are the feature maps split along the height and width directions respectively; F h and F ware 1×1 convolution transformation matrices along the height and width directions respectively; σ is the sigmoid activation function; g h and g w They are the attention weight maps obtained after convolution transformation and activation function in the height and width directions, respectively, and are used to adjust the weights of the input feature maps in the corresponding directions.

[0100] The obtained g h and g w The feature vectors that are expanded along the width and height directions to have the same size as the input feature map X are the CA weights. Multiplying the CA weights with the corresponding positions of the original input feature map can obtain an output with stronger feature expression capabilities:

[0101]

[0102] Among them, c represents the cth channel; i and j represent the horizontal and vertical coordinate positions respectively.

[0103] (2) Add a small target detection layer and an anchor frame for the small target detection layer. The specific steps are as follows:

[0104] Based on the two upsampling of the Neck network in YOLOv5, upsampling is performed again, and the obtained feature map is spliced ​​with the second layer of the Backbone network to obtain a large-resolution feature map for small target detection, and then normal downsampling operations are continued. Finally, the 21st, 24th, 27th, and 30th layer feature maps of the YOLOv5 network are used as the input of the Head;

[0105] Based on the virtual edge detection dataset, 12 anchor boxes are regenerated using the K-means clustering algorithm.

[0106] (3) Lightweight dynamic upsampling DySample is used to replace the bilinear interpolation upsampling in the network.

[0107] (4) CIoULoss (complete intersection-over-union loss) is changed to EIoULoss (efficient intersection-over-union loss), and the bounding box regression loss weight of the small object detection layer is appropriately increased. The calculation formula is as follows:

[0108]

[0109] Where IOU is the intersection-over-union ratio between the predicted box and the real box; ρ is the Euclidean distance between two points; b, w, h are the coordinates of the center point, width, and height of the predicted box; b gt 、w gt 、h gt w is the center point coordinates, width and height of the real frame; c 、h cis the width and height of the minimum outer rectangle of the predicted box and the real box; Loss EIOU is the EIoU loss value, which measures the error between the predicted bounding box and the true bounding box;

[0110] Change the confidence and classification loss from CrossEntropyLoss (cross entropy loss) to FocalLoss (focal loss), and the calculation formula is as follows:

[0111] Loss Focal (p t )=-α t (1-p t ) γ log(p t )

[0112] In the formula, p t Represents the model's predicted probability for the correct category; α t is a balancing factor used to balance the ratio of positive and negative samples; γ is an adjustment factor used to adjust the loss weight of easy-to-classify samples; Loss Focal (p t ) is the Focal Loss value, which measures the classification error of the model when dealing with unbalanced samples.

[0113] In an optional embodiment, a confidence threshold is set to filter the virtual edge detection results. In each virtual edge image to be detected, at most one detection result with the highest confidence and higher than the threshold is retained. The horizontal coordinates of the left and right sides of the target frame in the detection result can represent the positions of the left and right edges of the virtual edge. The absolute value of the subtraction between the two horizontal coordinates is the width of the virtual edge. Combined with the position information of the virtual edge image to be detected in the original image, the virtual edge coordinates are mapped to the original image to obtain the position and size information of the virtual edge in the original coated image, see Figure 4 .

[0114] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A virtual edge detection method for coated ceramic area based on deep learning, It is characterized in that The following steps are involved: Step S1, collecting coating image data through an industrial camera, and preprocessing the collected data, the preprocessing includes: grayscale, Gaussian blur and image scaling; Step S2, input the pre-processed coating image into the ceramic area detection model to obtain the location and type of the ceramic area; specifically: The lightweight object detection network YOLOv5-Lite is used as the ceramic area detection model, and the non-maximum suppression algorithm is used to remove redundant information in the model output to obtain the location and type of each ceramic area in the coating image; The types of the ceramic area include a ceramic area on the left side of the carbon film and a ceramic area on the right side of the carbon film; Step S3, constructing a roughly located virtual edge ROI region according to the position and type of the ceramic area, mapping the virtual edge ROI region back to the original image to obtain a virtual edge image to be detected, and performing local contrast enhancement on the virtual edge image to be detected; The specific implementation steps of the local contrast enhancement are: The virtual edge image to be detected is divided into several small blocks, and each small block is subjected to histogram equalization. The high-frequency grayscale value is limited by the contrast threshold, and the pixels exceeding the limit are evenly distributed to other parts of the histogram to limit the contrast of each small block. The calculation formula is: In the formula, S ij (k) indicates that at gray level k, the small block T ij The gray value after processing; L is the total number of gray levels; H ij (m) is the small piece T ij The number of pixels with a gray value of m; C is the contrast threshold, which is used to limit the high-frequency part of the histogram; For small block T ij The sum of the remaining histogram counts after subtracting the contrast threshold C from the histogram counts of all gray levels l in |T ij |For small pieces of T ij The total number of pixels, i.e. the area of ​​the small block; Use bilinear interpolation to smooth the boundaries between small blocks, eliminate the block effect, and smoothly merge all small blocks to form a complete contrast-enhanced virtual edge image to be detected; Step S4, improving the original YOLOv5 network by introducing an attention mechanism, adding a small target detection layer, changing the upsampling method and the loss function, obtaining an improved YOLOv5 network, training the improved YOLOv5 network using a virtual edge detection dataset, obtaining a virtual edge detection model, inputting the virtual edge image to be detected into the virtual edge detection model, and the virtual edge detection model outputs a virtual edge detection result; The virtual edge detection model is: marking the virtual edges in the virtual edge image to be detected, so that the left and right sides of the calibration frame coincide with the left and right sides of the virtual edge, obtaining a virtual edge data set, and inputting the virtual edge data set into the improved YOLOv5 model for training to obtain a virtual edge detection model; The improved YOLOv5 network includes the feature extraction network Backbone, the feature fusion network Neck and the detection head Head. The specific improvements to the YOLOv5 network include: (1) Introduce the coordinate attention mechanism CA into the C3 module of the feature extraction network Backbone network in the YOLOv5 network. The specific operation steps are as follows: The feature map obtained after the 3×3 convolution in the C3 module is globally averaged pooled using pooling kernels of size C′×1×W and C′×H×1 along the height and width directions, respectively, and the position information is encoded in a pair of direction-aware feature vectors, which is calculated as follows: Among them, C′, H, W represent the number of channels, height and width of the input feature map respectively; X∈R C′×H×W represents the input feature map; X h ∈R C′×1×W , X w ∈R C′×H×1 Represent the feature vectors after global average pooling in the horizontal and vertical directions respectively; After swapping the dimensions of one of the feature vectors, it is concatenated with the other feature vector and input into a transformation function shared by both. The calculation formula is: f=δ(F 1 (cat[X h ,X′ w ])) Among them, δ is a nonlinear activation function; F 1 is a 1×1 convolution transformation function; X′ W Represents X after the width and height dimensions are swapped w ; cat[X h ,X′ w ] indicates that X h and X′ w Concatenate the corresponding dimensions; f∈R C' / r×(H+W) is the intermediate feature vector that integrates the position information in both height and width directions, r is the maximum downsampling ratio, which is used to control the module size; Split f into two independent vectors f along the spatial dimension h ∈R C′ / r×H and f w ∈R C′ / r×W Input into two 1×1 convolution transformation functions F respectively h and F w In, we get: g h =σ(F h (f h )) g w =σ(F w (f w )) Among them, f h and f w are the feature maps split along the height and width directions respectively; F h and F w are 1×1 convolution transformation matrices along the height and width directions respectively; σ is the sigmoid activation function; g h and g w They are the attention weight maps obtained after convolution transformation and activation function in the height and width directions, respectively, which are used to adjust the weights of the input feature maps in the corresponding directions; The obtained g h and g w The feature vectors that are expanded along the width and height directions to have the same size as the input feature map X are the CA weights. Multiplying the CA weights by the corresponding positions of the original input feature map will give an output with stronger feature expression capabilities: Where, c represents the cth channel; i and j represent the horizontal and vertical coordinate positions respectively; (2) Add a small target detection layer and an anchor frame for the small target detection layer. The specific steps are as follows: Based on the 2-time upsampling of the feature fusion network Neck in YOLOv5, upsampling is performed again, and the obtained feature map is spliced ​​with the second layer of the feature extraction network Backbone to obtain a large-resolution feature map for small target detection, and then normal downsampling operations are continued. Finally, the 21st, 24th, 27th, and 30th layer feature maps of the YOLOv5 network are used as the input of the detection head Head; Based on the virtual edge detection dataset, 12 anchor boxes are regenerated using the K-means clustering algorithm; (3) Use lightweight dynamic upsampling DySample to replace the bilinear interpolation upsampling in the YOLOv5 network; (4) Adjust the loss function. The specific steps are as follows: The positioning loss is changed from complete intersection-over-union loss CIoU Loss to efficient intersection-over-union loss EIoU Loss, and the border regression loss weight of the small target detection layer is appropriately increased. The calculation formula is as follows: Where IOU is the intersection-over-union ratio between the predicted box and the real box; ρ is the Euclidean distance between two points; b, w, and are the coordinates of the center point, width, and height of the predicted box; b gt 、w gt 、h gt w is the center point coordinates, width and height of the real frame; c 、h c is the width and height of the minimum outer rectangle of the predicted box and the real box; Loss EIOU is the EIoU loss value, which measures the error between the predicted bounding box and the true bounding box; The confidence and classification loss are changed from cross entropy loss to focal loss, and the calculation formula is as follows: Loss Focal (p t )=-α t (1-p t ) γ log(p t ) In the formula, p t Represents the model's predicted probability for the correct category; α t is a balancing factor used to balance the ratio of positive and negative samples; γ is an adjustment factor used to adjust the loss weight of easy-to-classify samples; Loss Focal (p t ) is the Focal Loss value, which measures the classification error of the model when dealing with unbalanced samples; Step S5, analyzing the virtual edge detection result to obtain the position and size information of the virtual edge in the original coating image coordinate system.

2. The method for detecting virtual edges of coated ceramic areas based on deep learning according to claim 1, It is characterized in that In the step S3, the positional relationship between the ceramic area and the carbon film area is determined according to the position and type of the ceramic area, and the edges of the boundaries between the adjacent ceramic area and the carbon film area are used as a reference to expand to the left and right sides respectively, and the ROI area where the virtual edge is located is preliminarily located. The ROI area is mapped back to the original image for cropping. Since the virtual edge size only requires the calculation of width information but not height, the cropped image is scaled in the height direction while keeping the width unchanged to obtain the virtual edge image to be detected, ensuring that the virtual edge detection size has pixel-level accuracy while greatly improving the detection speed, and then the local contrast enhancement is performed on the virtual edge image to be detected as the input of the virtual edge detection model.

3. The method for detecting virtual edges of coated ceramic areas based on deep learning according to claim 2, It is characterized in that A confidence threshold is set to filter the virtual edge detection results, and at most one detection result with the highest confidence and higher than the threshold is retained in each virtual edge image to be detected; The horizontal coordinates of the left and right sides of the target frame in the detection result can represent the positions of the left and right edges of the virtual edge, and the absolute value of the difference between the two horizontal coordinates is the width of the virtual edge; Combined with the position information of the virtual edge image to be detected in the original image, the virtual edge coordinates are mapped to the original image to obtain the position and size information of the virtual edge in the original coated image.

Citation Information

Patent Citations

  • Video canvas boundary detection method and system, terminal equipment and storage medium

    CN113573137A

  • Vehicle-mounted target detection method based on improved YOLOv5

    CN115731533A

  • Improved YOLOv5s small target cloth flaw detection method

    CN117152484A

  • Small target detection method based on spatial context feature pyramid

    CN118212403A

  • Visual inspection method for corrosion of inner surface of cabin section of long and narrow closed space

    CN118446987A