Workpiece edge detection method and system based on deep learning fusion features

Abstract

The invention provides a deep learning fusion feature-based workpiece edge detection method. The method comprises the steps of obtaining a to-be-detected workpiece image; performing noise reduction processing on the to-be-detected workpiece image to obtain a noise-reduced image and a noise probability; inputting the noise reduction image and the noise probability into an edge enhancement model based on deep learning for identification, and obtaining an identified enhanced edge image and an edge probability; performing feature fusion on the noise probability and the edge probability by adopting a feature fusion algorithm based on machine learning, and converting a two-dimensional vector into a one-dimensional residual feature to form a residual vector; and inputting the enhanced edge image and the residual vector into an edge enhancement model based on deep learning for identification, carrying out clustering processing on the identified image, and outputting a target edge detection result and non-target features. The method can reduce the influence of irrelevant noise on target edge detection, and achieves the accurate recognition of the target edge.

Description

Technical Field

[0001] The present invention relates to a measuring tool detection system, and in particular to a workpiece edge detection method and system based on deep learning fusion features. Background Art

[0002] Edge detection technology is one of the important tasks in the field of machine vision and image processing. With the further development and widespread application of the technology, this technology plays an important role in application scenarios such as industrial automation, robotic vision, and digital manufacturing. Before the rapid development and comprehensive application of deep learning methods, the construction of traditional machine vision algorithms mainly relied on common image gradient algorithms and operators composed of filters, such as Sobel and Canny. These traditional common methods perform well in simpler scenarios. When the background complexity increases, the noise interference in the image is large, or the edge morphology of the workpiece to be measured is diverse, it is difficult for traditional algorithms to achieve ideal detection results.

[0003] With the rapid development of business scenarios with explosive growth in data volume and deep learning algorithms, edge detection algorithms based on deep learning have gradually gained another set of solutions. Taking convolutional neural networks (CNNs) as an example, this deep learning algorithm can form a powerful feature extraction capability through high-dimensional feature mapping, and improve learning ability by iteratively calculating loss functions based on a large amount of training data. Compared with traditional methods, this algorithm can achieve a more accurate and stable edge detection algorithm solution. As the basic network structure in deep learning, convolutional neural networks can realize local feature extraction of images based on multi-layer convolutional networks and pooling layers. The specific size of local features can be adjusted by parameters, and then the high-dimensional vectors composed of these features can be combined through fully connected layers to finally form feature parameters for edge detection. Based on the basic structure of convolutional neural networks, fully convolutional networks (FCNs) can further improve the accuracy of edge detection by fusing features at different scales. At the same time, in order to make up for the lack of feature data sets for similar scenes, generative adversarial networks can be used. By training generators and discriminators, the network can generate edge detection-related data that is closer to real scenes, or directly generate edge detection results. The above two methods can be tested separately, and the solution with higher accuracy is adopted. Summary of the invention

[0004] In response to the problems existing in the prior art, a workpiece edge detection method and system based on deep learning fusion features are provided. By integrating traditional machine vision and deep learning algorithms, the generalization and accuracy of the edge detection algorithm can be taken into account.

[0005] The technical solution adopted by the present invention is as follows: a workpiece edge detection method based on deep learning fusion features, comprising:

[0006] Acquire an image of a workpiece to be inspected;

[0007] Perform noise reduction processing on the workpiece image to be inspected to obtain a noise reduction image and noise probability;

[0008] The denoised image and noise probability are input into the edge enhancement model based on deep learning for recognition, and the recognized enhanced edge image and edge probability are obtained;

[0009] A feature fusion algorithm based on machine learning is used to fuse the noise probability and edge probability, converting the two-dimensional vector into a one-dimensional residual feature to form a residual vector;

[0010] The enhanced edge image and the residual vector are input into the edge enhancement model based on deep learning for recognition, the recognized image is clustered, and the target edge detection results and non-target features are output.

[0011] As a preferred solution, probability distribution is introduced during the noise reduction process to improve the robustness of the algorithm.

[0012] As a preferred solution, the denoising process is performed on the workpiece image to be detected to obtain the denoised image and the noise probability, which specifically includes:

[0013] Set the traversal window and divide it into several small images according to the traversal window and the preset step size;

[0014] Compare the similarities between each small image and classify the small images into categories;

[0015] For small images classified into the same category, the mean is calculated based on the grayscale value of each pixel in the small image, and then the difference between the grayscale value of each pixel in each small image and the mean is calculated. When abs(g1-g') / g'>0.5, it is judged that the difference is large, and the grayscale value of the pixel is replaced by the mean; where g1 is the grayscale value of the pixel in the original small image, and g' is the mean;

[0016] After processing all small images, the denoised image and noise probability are obtained.

[0017] As a preferred solution, comparing the similarity of each small image and classifying the small images into categories specifically includes:

[0018] A similarity percentage threshold p is set. When the similarity of the grayscale value distribution of the two images is greater than the p value, the two images are judged to be consistent. For large areas of background and blank, the threshold p is set to 1. For pixels near the edge target to be measured, the threshold p is set by traversing a straight line in the X and Y directions for judgment. When the grayscale difference between the upper and lower and left and right sides of the XY lines is low, it proves that the current range is consistent. All pixels less than the threshold p are classified into one category, and no noise reduction is performed.

[0019] As a preferred solution, in the process of inputting the denoised image and the noise probability into the edge enhancement model based on deep learning for recognition, the extracted noise probability features are used as the same-size channel expansion to increase the number of input channels.

[0020] As a preferred solution, the deep learning-based edge enhancement model performs inversion processing on positions with low edge probability, thereby avoiding the probability of noise positions being calculated as edges.

[0021] As a preferred solution, the feature fusion algorithm based on machine learning is used to fuse the noise probability and the edge probability, and the two-dimensional vector is converted into a one-dimensional residual feature to form a residual vector, which specifically includes:

[0022] Vectorize the noise probability and edge probability to obtain a two-dimensional vector feature;

[0023] Transform the two-dimensional vector features into one-dimensional vector features;

[0024] Calculate the variance expansion factors of all one-dimensional vector features and delete relevant features with values ​​greater than 10;

[0025] The remaining features are taken as the residual vector.

[0026] As a preferred solution, clustering the recognized image to output the target edge detection result and non-target features specifically includes:

[0027] For the recognized image, the constraint features of the target edge recognition are set to distinguish the target edge class features and non-edge class features, wherein the target edge features can be used to output the final edge detection results, and the non-edge class features can be used to supplement the noise feature extraction and deep learning edge recognition.

[0028] As a preferred solution, the constraint feature of the target edge recognition includes setting a priori threshold.

[0029] The second aspect of the present invention proposes a workpiece edge detection system based on deep learning fusion features, including a memory and a processor, wherein the memory stores a computer program corresponding to the workpiece edge detection method based on deep learning fusion features described in the first aspect that can be loaded and executed by the processor.

[0030] Compared with the prior art, the beneficial effects of adopting the above technical solution are:

[0031] (1) Perform noise processing on the image to be detected to reduce the impact of irrelevant noise on target edge detection.

[0032] (2) Implement edge feature enhancement based on deep learning algorithm. Use noise probability to enhance edge features before determining the detected edge, and classify non-edge features and noise into one category, both of which are non-target features.

[0033] (3) Based on high-dimensional residual features, the constraint features of target edge recognition are supplemented to realize edge recognition and clustering of non-edge features, preparing for subsequent algorithm iterations. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 This is a flow chart of the workpiece edge detection method based on deep learning fusion features proposed in the present invention. DETAILED DESCRIPTION

[0035] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar modules or modules with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be construed as limitations on the present application. On the contrary, the embodiments of the present application include all changes, modifications and equivalents that fall within the spirit and connotation of the appended claims.

[0036] In order to take into account the generalization and accuracy of the edge detection algorithm, the embodiment of the present invention provides a workpiece edge detection method based on deep learning fusion features, please refer to Figure 1 , the specific plan is as follows:

[0037] Step 1: Obtain the image of the workpiece to be inspected.

[0038] In this embodiment, the acquired workpiece image to be detected should have a relatively clear workpiece edge. If the edge is very blurred in the imaging effect or the brightness of the entire picture is very low, it is not included in the test range. For images with lower brightness, a common method can be used. The detection process proposed in this embodiment takes a grayscale image as an example, and the RGB image 3 channels are first converted into a grayscale image and then processed subsequently.

[0039] Step 2: Perform noise reduction processing on the workpiece image to be inspected to obtain a noise reduction image and noise probability.

[0040] The purpose of performing noise processing on the workpiece image to be detected is to reduce the influence of irrelevant noise on the target edge detection. In this embodiment, probability distribution is introduced to improve the robustness of the noise reduction algorithm. Specifically:

[0041] Set the traversal window to be a square with a side length of h (h>1) and a traversal step length of s (s>=1). Divide the complete image into several small images with a side length of h according to the traversal window and step length. By comparing the similarity of each small image, the small images are classified. The similarity percentage threshold p can be set. When the similarity of the gray value distribution of the two images is greater than the p value, the two images are judged to be consistent. When p is set to 1, the two small images are required to be completely consistent. This value can be used for processing large areas of background and blank space. However, for pixels near the edge target to be measured, the setting of the p value should be appropriately reduced to ensure that the imaging effect of the edge in different small images has a displacement deviation. The setting of the p value can be judged by traversing a straight line in the X and Y directions. When the grayscale difference between the upper and lower left and right sides of the XY lines is low, it proves that the current range is consistent. In order to consider the robustness of the model, the threshold p may be reduced in the later stage, and a high value is set in the early stage. The value can be determined according to the adaptation range of the training sample. All pixels less than the threshold p are classified into one category and no noise reduction is performed.

[0042] For small images classified into the same category, the mean is calculated according to the grayscale value of each pixel in the small image, and then the difference between the grayscale value of each pixel in each small image and the mean is calculated. The difference judgment principle is set. The grayscale value of the pixel in the original small image is g1, and the average value is g'. When abs(g1-g') / g'>0.5, it is judged that the difference is large, and the grayscale value of the pixel can be replaced by the mean.

[0043] After the image denoising is completed, two parts of data (n1, n2) are generated, n1 is the denoised image, and n2 is the noise probability, that is, the value of abs(g1-g') / g'.

[0044] Step 3: Input the denoised image and noise probability into the edge enhancement model based on deep learning for recognition, and obtain the recognized enhanced edge image and edge probability.

[0045] In this embodiment, a deep learning-based edge enhancement model HED (Deep Hull Edge Detection) is used for recognition. Different from the original model, before inputting the denoised image, this embodiment expands the extracted noise probability features as channels of the same size to increase the number of input channels. That is, if the image to be tested is a grayscale image, it will be converted into a 2-channel matrix.

[0046] This input will be used as the input of the HED model. The output of the HED model is the probability of whether each pixel value is an edge target. The positions with lower probability are inverted to avoid the probability of noise positions being calculated as edges. After the edge feature is enhanced through the HED model, two parts of data (d1, d2) are obtained, d1 is the enhanced edge image after recognition, and d2 is the edge probability.

[0047] The present invention realizes edge feature enhancement based on a deep learning algorithm, uses noise probability to enhance edge features before determining the detected edge, and classifies non-edge features and noise into one category, both of which are non-target features.

[0048] Step 4: Use a feature fusion algorithm based on machine learning to fuse the noise probability and edge probability, convert the two-dimensional vector into a one-dimensional residual feature, and form a residual vector.

[0049] In this embodiment, a machine learning algorithm is established to achieve feature fusion of noise probability and edge probability, and a combination scheme is proposed to achieve the selection of the optimal item:

[0050] 1) Feature vectorization of noise probability and edge probability, transforming the two-dimensional vector features into one-dimensional vector features.

[0051] 2) Calculate the variance expansion factors of all one-dimensional vector features and delete relevant features with values ​​greater than 10.

[0052] 3) Save the filtered features as residual vectors.

[0053] Step 5: Input the enhanced edge image and the residual vector into the edge enhancement model based on deep learning for recognition, perform clustering on the recognized image, and output the target edge detection results and non-target features.

[0054] In this embodiment, the deep learning-based edge enhancement model established in step 3 is used again for detection, and the residual vector is used as the residual input by means of the fully connected layer to participate in the final detection of whether each pixel belongs to the edge.

[0055] Based on high-dimensional residual features, the constraint features of target edge recognition are supplemented to realize the recognition of target edge features and clustering of non-edge features, preparing for subsequent algorithm iterations. Specifically, target edge features can be used to output the final edge detection results, and non-edge features can be used to supplement noise feature extraction and deep learning edge extraction.

[0056] In one embodiment, the constraint feature for target edge recognition may be a priori threshold.

[0057] An embodiment of the present invention also proposes a workpiece edge detection system based on deep learning fusion features, including a memory and a processor, wherein the memory stores a computer program corresponding to the aforementioned workpiece edge detection method based on deep learning fusion features that can be loaded and executed by the processor.

[0058] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer readable medium, and the computer program includes a program code for executing the method shown in the flowchart.

[0059] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0060] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0061] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0062] As another aspect, the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the workpiece edge detection method based on deep learning fusion features described in the above embodiment.

[0063] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the workpiece edge detection method based on deep learning fusion features described in the above embodiment.

[0064] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0065] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0066] For those skilled in the art, the specific meanings of the above terms in the present invention can be understood in specific situations; the drawings in the embodiments are used to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0067] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A workpiece edge detection method based on deep learning fusion features, characterized in that: include: Acquire an image of a workpiece to be inspected; Perform noise reduction processing on the workpiece image to be inspected to obtain a noise reduction image and noise probability; The denoised image and noise probability are input into the edge enhancement model based on deep learning for recognition, and the recognized enhanced edge image and edge probability are obtained; A feature fusion algorithm based on machine learning is used to fuse the noise probability and edge probability, converting the two-dimensional vector into a one-dimensional residual feature to form a residual vector; The enhanced edge image and the residual vector are input into the edge enhancement model based on deep learning for recognition, the recognized image is clustered, and the target edge detection results and non-target features are output.

2. The workpiece edge detection method based on deep learning fusion features according to claim 1 is characterized in that: In the process of noise reduction, probability distribution is introduced to improve the robustness of the algorithm.

3. The workpiece edge detection method based on deep learning fusion features according to claim 1 or 2, characterized in that: The denoising process is performed on the workpiece image to be detected to obtain the denoised image and the noise probability, specifically including: Set the traversal window and divide it into several small images according to the traversal window and the preset step size; Compare the similarities between each small image and classify the small images into categories; For small images classified into the same category, the mean is calculated based on the grayscale value of each pixel in the small image, and then the difference between the grayscale value of each pixel in each small image and the mean is calculated. When abs(g1-g') / g'>0.5, it is judged that the difference is large, and the grayscale value of the pixel is replaced by the mean; where g1 is the grayscale value of the pixel in the original small image, and g' is the mean; After processing all small images, the denoised image and noise probability are obtained.

4. The workpiece edge detection method based on deep learning fusion features according to claim 3 is characterized in that: The comparing the similarity of each small image and classifying the small images into categories specifically includes: A similarity percentage threshold p is set. When the similarity of the grayscale value distribution of the two images is greater than the p value, the two images are judged to be consistent. For large areas of background and blank, the threshold p is set to 1. For pixels near the edge target to be measured, the threshold p is set by traversing a straight line in the X and Y directions for judgment. When the grayscale difference between the upper and lower and left and right sides of the XY lines is low, it proves that the current range is consistent. All pixels less than the threshold p are classified into one category, and no noise reduction is performed.

5. The workpiece edge detection method based on deep learning fusion features according to claim 1 is characterized in that: In the process of inputting the denoised image and the noise probability into the edge enhancement model based on deep learning for recognition, the extracted noise probability features are used as the same-size channel expansion to increase the number of input channels.

6. The workpiece edge detection method based on deep learning fusion features according to claim 5 is characterized in that: The deep learning-based edge enhancement model performs inversion processing on positions with low edge probabilities, thereby avoiding the probability of noise positions being calculated as edges.

7. The workpiece edge detection method based on deep learning fusion features according to claim 1 is characterized in that: The feature fusion algorithm based on machine learning is used to fuse the noise probability and the edge probability, and the two-dimensional vector is converted into a one-dimensional residual feature to form a residual vector, which specifically includes: Vectorize the noise probability and edge probability to obtain a two-dimensional vector feature; Transform the two-dimensional vector features into one-dimensional vector features; Calculate the variance expansion factors of all one-dimensional vector features and delete relevant features with values ​​greater than 10; The remaining features are taken as the residual vector.

8. The workpiece edge detection method based on deep learning fusion features according to claim 1 is characterized in that: The clustering process of the recognized image to output the target edge detection result and the non-target features specifically includes: For the recognized image, the constraint features of the target edge recognition are set to distinguish the target edge class features and non-edge class features, wherein the target edge features can be used to output the final edge detection results, and the non-edge class features can be used to supplement the noise feature extraction and deep learning edge recognition.

9. The workpiece edge detection method based on deep learning fusion features according to claim 8 is characterized in that: The constraint feature of the target edge recognition includes setting a priori threshold.

10. A workpiece edge detection system based on deep learning fusion features, characterized in that: It comprises a memory and a processor, wherein the memory stores a computer program corresponding to the workpiece edge detection method based on deep learning fusion features as described in any one of claims 1 to 9, which can be loaded by the processor and executed.

Images