Workpiece saliency edge detection method, product, medium, system and equipment
By adopting the workpiece saliency edge detection method in industrial quality inspection, using frequency domain analysis and dual-branch backbone network technology, the problem of difficulty in identifying saliency edge defects in the existing technology is solved, and a higher accuracy detection effect is achieved.
Patent Information
- Application Number
- CN202411675134.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-21
- Publication Date
- 2025-02-28
AI Technical Summary
In industrial quality inspection, it is difficult for the prior art to effectively identify the significant edge defects of the workpiece through deep learning object detection algorithms, especially in the absence of sufficient real sample data, and the existing edge detection algorithms cannot finely distinguish texture areas and saliency edges.
A workpiece saliency edge detection method is adopted. By acquiring the workpiece images, the depth and depth hierarchical features are extracted, and converted into high-frequency components and low-frequency components in the frequency domain. The high-frequency components are aligned and fused through octave convolution and cavity convolution techniques to form a dual-branch backbone network to expand the receptive field and achieve fine detection of the saliency edge.
The accuracy of workpiece salient edge detection is improved, and the texture area and distinct edge can be more finely divided, meeting the needs of fine inspection in industrial quality inspection, and improving the detection accuracy by about 7.74%.
Smart Images
Figure CN119540201A_ABST
Abstract
Description
Technical Field
[0001] The present invention mainly relates to the technical field of workpiece defect recognition, and specifically relates to a workpiece significant edge detection method, product, medium, system and equipment. Background Art
[0002] In the field of industrial quality inspection, it is necessary to frequently detect defects in various parts. In order to detect such defects in real time, the industry usually uses a deep learning-based target detection algorithm, that is, to detect defects as a target category. Common target detection algorithms include the YOLO series algorithm and the DETR algorithm based on the transFromer class. The above algorithms all rely on a large amount of real sample data. However, in practical industrial applications, it is difficult to collect enough real samples to cover various defect forms in most cases, which makes it difficult for the target detection algorithm model trained on a specific data set to be universal. In order to deal with this problem, it is necessary to combine the edge detection algorithm based on deep learning. This type of algorithm has better universality because it detects relatively low-level features such as edge information. Basically, training on a public data set can meet certain universality requirements. However, in some scenarios where only significant edges need to be detected, this type of algorithm cannot distinguish the texture and significant edges of the target well, and uniformly treats them as edges to be processed, which cannot meet more refined needs.
[0003] like Figure 1 The figure shows the model structure of the existing edge detection algorithm DexiNed (Dense Extreme Inception Network for Edge Detection). After the backbone network extracts six different levels of feature maps, they enter the upsampling network module respectively, and then fuse them and finally output the results. The entire process of the above algorithm is carried out in the RGB domain. There is also the problem of being unable to distinguish between texture areas and significant edge areas. When only significant edge defects need to be detected, the algorithm will detect texture areas as defects. At the same time, edge detection requires a larger receptive field. The above algorithm lacks a large receptive field because it uses traditional convolutions and is not suitable for significant edge detection. Summary of the invention
[0004] In view of the technical problems existing in the prior art, the present invention provides a method, product, medium, system and equipment for detecting a significant edge of a workpiece with high detection accuracy.
[0005] In order to solve the above technical problems, the technical solution proposed by the present invention is: A method for detecting significant edges of a workpiece comprises the following steps: 1) Obtain the image of the workpiece to be tested, extract features from the image according to the depth and shallowness, and obtain features at different levels; 2) Convert features at different levels into high-frequency and low-frequency components in the frequency domain; 3) Align and fuse the high-frequency components at different levels to obtain the edge image.
[0006] Preferably, in step 2), features at different levels are converted into high-frequency components and low-frequency components in the frequency domain through octave convolution.
[0007] Preferably, the calculation formula corresponding to the octave convolution is:
[0008]
[0009] in represents the input of the octave convolution, Represents input The high frequency component of Represents input The low-frequency component of represents the high frequency component of the octave convolution output, represents the low-frequency component of the octave convolution output, ( ) represents the convolution operation, W is the weight parameter, Represents the weight parameter corresponding to the convolution operation of the high-frequency component, Represents the convolution parameters for inter-frequency communication from low-frequency components to high-frequency components, represents the upsampling operation, Represents a downsampling pooling operation.
[0010] Preferably, in step 1), an edge detection model is constructed, and a receptive field enhancement module consisting of a dilated convolution is added to the shallow layer of the backbone network of the edge detection model to construct a dual-branch backbone network, and features of the image are extracted through the dual-branch backbone network.
[0011] Preferably, the calculation formula for the receptive field n of the dilated convolution is:
[0012] Among them, n is the size of the receptive field, k is the size of the convolution kernel, and d is the dilation rate.
[0013] The present invention also discloses a workpiece salient edge detection system, comprising: The feature extraction module is used to obtain the image of the workpiece to be tested, extract features from the image according to the depth level, and obtain features at different levels; Feature analysis module, used to convert features at different levels into high-frequency components and low-frequency components in the frequency domain; The feature fusion module is used to align and fuse high-frequency components at different levels to obtain an edge image.
[0014] Preferably, the feature extraction module further includes a receptive field enhancement module for expanding the receptive field of edge features to extract contextual information of image edges.
[0015] The present invention further discloses a computer program product, comprising a computer program, wherein the computer program executes the steps of the method described above when executed by a processor.
[0016] The present invention also discloses a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described above are executed.
[0017] The present invention further discloses a computer device, comprising a memory and a processor connected to each other, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the method described above are executed.
[0018] Compared with the prior art, the advantages of the present invention are: The present invention adds frequency domain analysis to the original edge detection algorithm, decomposes image features into low-frequency components and high-frequency components (wherein there are a large number of high-frequency components in the significant edge region of the image, and the texture corresponds to lower-frequency components), and distinguishes between the significant edge region with drastic changes and the texture region with slower changes in the image through the high-frequency components and the low-frequency components, and then detects the edge from the high-frequency feature components, realizes the detection of significant edges, and meets the needs of refined scenarios; at the same time, a branch structure composed of void convolution is added to the shallow layer of the backbone network to form a dual-branch backbone network, wherein the newly added branch is mainly used to expand the algorithm receptive field and extract richer contextual information of the image edge, and the original structure branch part is mainly used to extract the spatial information details of the image, and the two work together to realize accurate detection of significant edges. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 The network architecture diagram of the existing edge detection model.
[0020] Figure 2 FIG. 4 is a network architecture diagram of an edge detection model in an embodiment of the present invention.
[0021] Figure 3 It is a double-branch structure diagram of the backbone network in the edge detection model of the present invention.
[0022] Figure 4 FIG. 4 is a flow chart of an edge detection method in an embodiment of the present invention. DETAILED DESCRIPTION
[0023] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.
[0024] like Figure 2-Figure 4 As shown, the workpiece salient edge detection method provided by the embodiment of the present invention comprises the steps of: 1) Obtain the image of the workpiece to be tested, extract features from the image according to the depth and shallowness, and obtain features at different levels; 2) Convert features at different levels into high-frequency and low-frequency components in the frequency domain; 3) Align and fuse the high-frequency components at different levels to obtain an edge image, thereby realizing the detection of significant edges of the workpiece.
[0025] The workpiece significant edge detection method of the present invention addresses the problem that the original edge detection algorithm DexiNed cannot accurately distinguish between texture areas and significant edge areas. By introducing frequency domain analysis to convert features of different levels in the image into high-frequency components and low-frequency components in the frequency domain, the image feature signals are distinguished and processed from the perspective of the frequency domain, which can effectively achieve the distinction between texture areas and significant edge areas.
[0026] Specifically, according to digital image correlation theory, RGB image signals can be converted into frequency domain signals. Low-frequency components mainly correspond to the parts of the image such as color, grayscale value, texture, etc. that change slowly, while high-frequency components correspond to areas where grayscale values change dramatically, such as the significant edge areas of objects. In the scenario of industrial defect detection, serious defects such as cracks will inevitably form significant edge areas. The image signal of this part must be a high-frequency component, while other defects such as texture will not change so dramatically, so the contained frequency domain components are mainly low-frequency components.
[0027] Based on the above theoretical foundation, the present invention adds frequency domain analysis to the original edge detection algorithm, decomposes the image features into low-frequency components and high-frequency components (wherein the significant edge area of the image has a large number of high-frequency components, while the texture corresponds to the lower-frequency components), and uses the high-frequency components and low-frequency components to distinguish between the significant edge areas with drastic changes and the texture areas with slower changes in the image, and then detects the edges from the high-frequency feature components, thereby realizing the detection of significant edges and meeting the needs of refined scenarios.
[0028] Furthermore, in order to realize the frequency division processing of image features, the present invention performs frequency domain analysis based on octave convolution. Octave convolution is a convolution algorithm specifically for processing high-frequency and low-frequency signals of images. Its original design is to reduce the resolution of low-frequency feature maps, reduce the redundancy of low-frequency features, expand the network receptive field, and achieve the goal of maintaining accuracy while reducing the amount of calculation.
[0029] Specifically, the input and output of the octave convolution are expressed as:
[0030]
[0031] in represents the input of the octave convolution, Represents input The high frequency component of Represents input The low-frequency component of represents the high frequency component of the octave convolution output, Represents the low-frequency component of the output, ( ) represents the convolution operation, W is the weight parameter, Represents the weight parameter corresponding to the convolution operation of the high-frequency component, Represents the convolution parameters for inter-frequency communication from low-frequency components to high-frequency components, Indicates upsampling operation with a sampling factor of 2. Represents a downsampling pooling operation with a downsampling factor of 2.
[0032] Based on the above octave convolution operation, construct Figure 2 The frequency domain processing module in the dotted box. Figure 2 As shown, , respectively represent the feature maps from the three levels of the backbone network, which are divided into high-frequency components and low-frequency components after octave convolution. Since the object applicable to the present invention is the significant edge in defect detection, this area is manifested as a high-frequency component with drastic changes in the frequency domain, so only the high-frequency component of the octave convolution output is taken. As input for the next step; After the high-frequency components of feature maps from different feature levels are extracted, up-sampled, multiplied, and convolved, the high-frequency information of different levels is fused to finally output the result.
[0033] Furthermore, in order to expand the receptive field and improve the accuracy, the present invention designs a dual-branch structure of the backbone network based on dilated convolution, such as Figure 3 As shown in the figure, atrous convolution adds a dilation rate to the standard convolution, so that a small convolution kernel can also obtain the receptive field effect of a large convolution kernel without increasing the amount of calculation.
[0034] The calculation formula for the receptive field n of the dilated convolution is:
[0035] Among them, n is the size of the receptive field, k is the size of the convolution kernel, and d is the dilation rate.
[0036] Usually, dilated convolution is used in higher layers of the algorithm model to help the algorithm obtain high-level semantic information. If it is only used in shallow layers and simply replaces the traditional convolution, it may cause the algorithm accuracy to decrease due to the loss of details. Based on dilated convolution, the present invention adds a receptive field enhancement module (such as dilated convolution) composed of dilated convolution in the shallow layer of the backbone network with rich edge information. Figure 3 The dotted box in the middle) is built together with the original branch Figure 3 The dual-branch structure shown in the figure. The newly added branch is mainly used to expand the receptive field of edge features to extract the contextual information of the image edge, and the original branch is used to extract the spatial information details of the image. The two work together to improve the detection accuracy.
[0037] The atrous convolution adds a dilation rate to the standard convolution to control the distance between the convolution kernel and the data during calculation, so that a small convolution kernel can obtain a receptive field similar to that of a large convolution kernel without increasing the amount of computation.
[0038] The present invention introduces a dual-branch structure by introducing a dilated convolution, so that the algorithm model can obtain sufficient receptive field and context information while maintaining feature details, which can effectively improve the detection accuracy of salient areas.
[0039] like Figure 2 Shown is the overall structure of the model of the present invention, Figure 2 The dotted box is the newly added frequency domain processing module, and the backbone network is Figure 3 The dual-branch structure shown in the figure; the core part of the frequency domain processing module is the octave convolution, which is used to convert the image features in the RGB domain into high and low frequency components in the frequency domain; the role of cat fusion is to combine features from different feature levels The role of upsampling is to align the scales of feature maps at different levels; the role of traditional convolution is to adjust the number of channels.
[0040] When a sample image of a defect to be detected is input, after feature extraction by the backbone network, three feature maps of different layers are output. ; Among them, from the shallow It mainly contains lower-level feature information such as edges, contours, and colors; Mainly includes feature information from the middle layer; Has more abstract semantic information; After octave convolution, they are decomposed into high-frequency components and low-frequency components. Since the significant edge area is concentrated in the high-frequency component, only the high-frequency component of the octave convolution output is taken. , ; After the high-frequency components are upsampled and fused, they are adjusted through the traditional 1×1 convolution to obtain the final output.
[0041] In the actual application process, the edge detection model of the present invention is built and trained in advance, and the specific process is as follows: Step 1: Dataset annotation: annotate images containing target defects or other significant edge information as training samples and test samples; Step 2: Network model construction: Figure 2 The edge detection network model structure shown; Step 3: Network model training: After building the network model, configure hyperparameters such as learning rate and batch size, divide the labeled data set in step 1 into training set and test set in a ratio of 7:3, and select a suitable optimizer to train the network model; Step 4: Forward reasoning: Select the model with the highest accuracy in the test set from the trained network models as the best model, input the image data to be detected into the model, and output the detection results.
[0042] Experimental verification: In the defect detection dataset, the test results are shown in Table 1, where the experimental environment is Ubuntu 20.0 and the GPU is Nvidia 3060: Table 1. Experimental data results
[0043] As shown in Table 1, due to the addition of the receptive field enhancement module and the frequency domain processing module, the algorithm improves the detection accuracy of the salient edge and accurately separates the high-frequency components of the image. The algorithm only focuses on the characteristics of the high-frequency components and is more sensitive to the high-frequency component areas with drastic changes caused by the salient edges. Therefore, it can more accurately distinguish between salient edges and ordinary textures, and the detection accuracy (AP) is improved by about 7.74%.
[0044] The embodiment of the present invention further provides a workpiece salient edge detection system, comprising: The feature extraction module is used to obtain the image of the workpiece to be tested, extract features from the image according to the depth level, and obtain features at different levels; Feature analysis module, used to convert features at different levels into high-frequency components and low-frequency components in the frequency domain; The feature fusion module is used to align and fuse high-frequency components at different levels to obtain an edge image.
[0045] Furthermore, a receptive field enhancement module is added to the feature extraction module to expand the receptive field of the edge feature to extract the contextual information of the image edge.
[0046] The embodiment of the present invention further discloses a computer program product, including a computer program, which executes the steps of the above method when executed by a processor.
[0047] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described above are executed.
[0048] The embodiment of the present invention further discloses a computer device, including a memory and a processor connected to each other, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the above method are executed.
[0049] The detection system, product, medium and equipment of the present invention correspond to the above method and also have the advantages described in the above method.
[0050] The present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by hardware related to computer program instructions. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiment can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. Computer-readable storage media include: any entity or device that can carry computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. The memory is used to store computer programs and / or modules. The processor implements various functions by running or executing computer programs and / or modules stored in the memory, and calling data stored in the memory. The memory may include a high-speed random access memory and may also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.
[0051] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.
Claims
1. A method for detecting a significant edge of a workpiece, characterized in that: Includes steps: 1) Obtain the image of the workpiece to be tested, extract features from the image according to the depth and shallowness, and obtain features at different levels; 2) Convert features at different levels into high-frequency and low-frequency components in the frequency domain; 3) Align and fuse the high-frequency components at different levels to obtain the edge image.
2. The method for detecting significant edges of a workpiece according to claim 1, characterized in that: In step 2), the features at different levels are converted into high-frequency components and low-frequency components in the frequency domain through octave convolution.
3. The workpiece salient edge detection method according to claim 2, characterized in that: The calculation formula corresponding to the octave convolution is: in represents the input of the octave convolution, Represents input The high frequency component of Represents input The low-frequency component of represents the high frequency component of the octave convolution output, represents the low-frequency component of the octave convolution output, ( ) represents the convolution operation, W is the weight parameter, Represents the weight parameter corresponding to the convolution operation of the high-frequency component, Represents the convolution parameters for inter-frequency communication from low-frequency components to high-frequency components, represents the upsampling operation, Represents a downsampling pooling operation.
4. The workpiece salient edge detection method according to claim 1, 2 or 3, characterized in that: In step 1), an edge detection model is constructed, and a receptive field enhancement module consisting of a hole convolution is added to the shallow layer of the backbone network of the edge detection model to construct a double-branch backbone network, and feature extraction of the image is performed through the double-branch backbone network.
5. The workpiece salient edge detection method according to claim 4, characterized in that: The calculation formula for the receptive field n of the dilated convolution is: Among them, n is the size of the receptive field, k is the size of the convolution kernel, and d is the dilation rate.
6. A workpiece salient edge detection system, characterized in that: include: The feature extraction module is used to obtain the image of the workpiece to be tested, extract features from the image according to the depth level, and obtain features at different levels; Feature analysis module, used to convert features at different levels into high-frequency components and low-frequency components in the frequency domain; The feature fusion module is used to align and fuse high-frequency components at different levels to obtain an edge image.
7. The workpiece salient edge detection system according to claim 6, characterized in that: The feature extraction module also includes a receptive field enhancement module, which is used to expand the receptive field of the edge feature to extract contextual information of the image edge.
8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are performed.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program performs the steps of the method according to any one of claims 1 to 5.
10. A computer device comprising a memory and a processor connected to each other, wherein a computer program is stored in the memory, characterized in that: When the computer program is executed by a processor, the computer program performs the steps of the method according to any one of claims 1 to 5.