Part circle detection method and device, electronic equipment and storage medium

Abstract

The present application relates to a kind of part circle detection method, device, electronic equipment and storage medium.The part circle detection method described in the present application includes: obtaining the part picture to be detected, the part picture contains several round holes;The part picture is input into the target detection network model trained, obtains the round hole result atlas corresponding to the part, the round hole result atlas includes the single round hole graph corresponding to all round holes in the part picture;The round hole result atlas is input into the semantic segmentation network model trained, calculates the contour segmentation result heat map corresponding to each round hole, obtains the circle detection result of the part picture.The part circle detection method described in the present application is trained for the data set of the complex scene of part, so that the present application can meet most part detection scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, in particular to a part circle detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] The traditional circle detection method first needs to detect the edge of the circle from the image, and then calculates the parameters of the circle according to the circular contour. The common parameter calculation method is the circle detection method based on Hough transform, which uses the mapping from image space to parameter space for prediction. The principle of the random circle detection method is to randomly select four points in the point set to verify whether they can form a circle. However, the traditional circle detection method is easily contaminated by noise and has large calculation amount, and cannot be applied to complex production environment.

[0003] In recent years, with the development of artificial intelligence technology, image feature detection technology begins to use deep neural network to extract rich image features. In 2015, Xe combined the fully convolutional network with the deep supervision network to propose the HED algorithm, which is a milestone in the edge detection of deep learning. According to the feature output of different scales, a linear classifier is used for prediction to realize end-to-end training and prediction. After that, many excellent feature detection and semantic segmentation methods based on deep learning appeared. In 2021, Zhang proposed a coin detection method combining target detection and pose estimation. First, the target detection is used to identify the coin, then the pose estimation method is used to detect the edge of the circle, and then the K-means clustering method is used to filter out the noise points. After that, the optimal circle is obtained through the iterative selection method, and finally the cropped circle is mapped back to the original image to obtain the detected circle of the original image.

[0004] However, the existing detection technology can only identify the overall contour of the object and cannot detect the specific small features of the object, and at present there is still a lack of a mature end-to-end circle detection method based on deep learning. SUMMARY

[0005] Therefore, the purpose of the present application is to provide a part circle detection method, device, electronic equipment and storage medium, which can meet most part detection scenes by constructing a data set for the complex scene of parts and targeted training.

[0006] In a first aspect, the present application provides a part circle detection method, comprising the following steps:

[0007] Obtaining a part picture to be detected, the part picture containing a plurality of circular holes;

[0008] inputting the part picture into the trained target detection network model to obtain a circle hole result atlas corresponding to the part, the circle hole result atlas including a single circle hole graph corresponding to all circle holes in the part picture;

[0009] inputting the circle hole result atlas into a trained semantic segmentation network model to calculate a contour segmentation result heat map corresponding to each circle hole, and obtaining a circle detection result of the part picture.

[0010] Further, the method further comprises the following steps:

[0011] For the contour segmentation result heat map corresponding to each circle hole, a random sample consensus method is used to obtain the circle parameters corresponding to the circle hole.

[0012] Further, the target detection network model is a YOLOv5 model.

[0013] Further, the prior box size of the YOLOv5 is [10:13, 26:30, 33:29], [55:61, 62:54, 108:119], [116:110, 190:198, 338:326].

[0014] Further, the semantic segmentation network model is a BiSeNet network model.

[0015] Further, the activation function of the BiSeNet network model is a Leaky ReLU activation function.

[0016] Further, for the contour segmentation result heat map corresponding to each circle hole, a random sample consensus method is used to obtain the circle parameters corresponding to the circle hole, including:

[0017] S11: For each corresponding contour segmentation result heat map, three points are randomly selected from the point set to calculate a model of a circle;

[0018] S12: Substituting all points in the point set into the model, the number of inliers satisfying the threshold requirement is calculated;

[0019] S13: Comparing the number of inliers of the current model with the number of inliers of the current best model, recording the model parameters and the number of inliers of the maximum number of inliers;

[0020] S14: Repeating S11-S13 until the number of iterations reaches a specified value or the number of inliers of the current model is greater than a set threshold. Wherein:

[0021]

[0022] P = 1 - (1 - t n )k

[0023]

[0024] wherein t is the ratio of inliers in the one iteration model, Nin is the number of inliers, Nout is the number of outliers, n is the number of points used in each iteration, k is the number of iterations, and P is the probability of sampling n inliers in k iterations.

[0025] In a second aspect, the present application further provides a part circle detection device, comprising:

[0026] a part picture acquisition module, configured to acquire a part picture to be detected, the part picture containing a plurality of circular holes;

[0027] a circular hole result atlas acquisition module, configured to input the part picture into a trained target detection network model to obtain a circular hole result atlas corresponding to the part, the circular hole result atlas including a single circular hole picture corresponding to each circular hole in the part picture;

[0028] a detection result acquisition module, configured to input the circular hole result atlas into a trained semantic segmentation network model to calculate a contour segmentation result heat map corresponding to each circular hole, and obtain a circle detection result of the part picture.

[0029] In a third aspect, the present application further provides an electronic device, comprising:

[0030] at least one memory and at least one processor;

[0031] the memory, configured to store one or more programs;

[0032] when the one or more programs are executed by the at least one processor, the at least one processor implements the steps of the part circle detection method according to any one of the first aspect of the present application.

[0033] In a fourth aspect, the present application further provides a computer readable storage medium,

[0034] the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the part circle detection method according to any one of the first aspect of the present application.

[0035] The application provides a part circle detection method and device, electronic equipment and a storage medium. The part circle detection method comprises the following steps: firstly, recognizing a circle in a part by using a target detection method and obtaining contour information of the circle by using a semantic segmentation method; and finally, estimating a circle parameter by using a random sample consensus method. The circle in the part is detected, a dataset for the complex part scene is constructed for targeted training, so that the application can meet most part detection scenes. Since the deep learning has strong feature learning capability, the deep learning can be used to realize high-precision circle detection, effectively solve the problems of part occlusion, shadow and false circle interference, and is more stable and accurate than traditional laser detection and more efficient and convenient than manual detection. Compared with other deep learning detection methods, the application uses a multi-step detection method of coarse detection and fine detection to simplify the multi-circle detection task into a single-circle detection task, reduces the parameter quantity and calculation quantity of training, so that the network model of the application can obtain high accuracy with only a small sample dataset, and promotes the development of deep learning part circle detection research.

[0036] For better understanding and implementation, the application will be described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 A flowchart of a part circle detection method used in a preferred embodiment of the application;

[0038] Figure 2 A step schematic diagram of a part circle detection method provided by the application;

[0039] Figure 3 A structure schematic diagram of a part circle detection device provided by the application. DETAILED DESCRIPTION

[0040] To make the purpose, technical scheme and advantages of the application clearer, the application embodiments will be further described in detail below with reference to the accompanying drawings.

[0041] It should be clear that the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application embodiments, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application embodiments.

[0042] The terms used in the application embodiments are only for the purpose of describing the specific embodiments, and are not intended to limit the application embodiments. The singular forms "a", "an" and "the" used in the application embodiments and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein means and includes any or all possible combinations of one or more associated listed items.

[0043] The following description refers to the accompanying drawings. In the following description, identical or similar elements in different drawings are denoted by the same reference numerals, unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Instead, they are merely examples consistent with some aspects of the present disclosure as detailed in the appended claims. In the description of the present disclosure, it should be understood that the terms "first", "second", "third", etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor can they be understood to indicate or imply relative importance. For those skilled in the art, the specific meaning of the above terms in the present disclosure can be understood according to the specific circumstances.

[0044] In addition, in the description of the present disclosure, "multiple" means two or more, unless otherwise indicated. The association between the associated objects is described, and there are three possible relationships, for example, A and / or B, which can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after it.

[0045] To solve the problems in the background art, the present disclosure provides a circle detection method combined with target detection semantic segmentation, as shown in the flowchart Figure 1 First, a target detection dataset about the part is constructed, and then a target detection network model is trained using the dataset to obtain a target detection network model for the part, which realizes rough detection of the circle in the part picture. After obtaining the rough detection picture containing a single circle, the rough detection picture is used for semantic segmentation annotation to construct a semantic segmentation dataset about the circle. After training the semantic segmentation network model using the segmentation dataset, a semantic segmentation model for the circle features is obtained. The model is used for semantic segmentation and fine detection of the rough detection picture. After obtaining the fine detection circle contour heat map, random sampling algorithm is used to sample the circle contour to calculate the circle parameters, thereby obtaining the diameter and center coordinates of the circle, and realizing end-to-end circle detection.

[0046] Based on the trained target detection network model and semantic segmentation network model, the present disclosure provides a part circle detection method, as shown in Figure 2 The method comprises the following steps:

[0047] S01: Obtain a part picture to be detected, which contains a plurality of circular holes.

[0048] In a preferred embodiment, the part picture is pre-processed to 640x640 pixels, and the picture smaller than 640 pixels is filled with a white background.

[0049] S02: inputting the part picture into the trained target detection network model to obtain a circle hole result set corresponding to the part, the circle hole result set including a single circle hole corresponding to all circle holes in the part picture.

[0050] In a preferred embodiment, the target detection network model is a YOLOv5 model.

[0051] The purpose of rough detection is to simplify multiple circle detection tasks of a part picture into a single circle detection task, thereby reducing the calculation amount for subsequent fine detection tasks. The target detection model used in the present application is a YOLOv5 model, and there are four major versions of YOLOv5 so far. Since the rough detection has a lower requirement for accuracy, the YOLOv5s version with the smallest depth and feature map width is used in the present project. Considering that the target to be detected is a circle, the prior box size of YOLOv5 is modified according to the characteristic that the length and width of a circle are equal, so that the aspect ratio of the prior box is closer to 1:1 to conform to the characteristics of the circle. The original three sizes [10:13, 16:30, 33:23], [30:61, 62:45, 59:119], and [116:90, 156:198, 373:326] are modified to [10:13, 26:30, 33:29], [55:61, 62:54, 108:119], and [116:110, 190:198, 338:326].

[0052] Suppose that the circle hole result set obtained after the picture is detected by YOLOv5s is D, and the circle holes in D are located at the center of the picture and the diameter of the circle occupies about 2 / 3 to 4 / 5 of the picture length. This step converts the entire part picture into a small picture containing only a single circle hole, greatly reducing the picture size for the subsequent semantic segmentation and reducing the computational burden.

[0053] Preferably, the training data set of the target detection model comprises:

[0054] The front part pictures with circle holes are collected from the network, all the pictures are processed to 640x640 pixels, the pictures smaller than 640 pixels are filled with white background, and then the LabelImg labeling software is used for labeling. Note that LabelImg is a visual image labeling tool, and the data set required by the mainstream target detection network can be labeled by using this software tool to label the target in the image.

[0055] Finally, a part picture data set with 160 labeled pictures is obtained. Since the rough detection task only needs to detect the approximate position of the circle hole in the part without accurately detecting the edge of the circle hole, the rough detection task has a low requirement for the data set.

[0056] Using the above labeled part picture dataset, the training of the target detection model can be completed.

[0057] S03: inputting the round hole result picture set into the trained semantic segmentation network model, calculating the contour segmentation result heat map corresponding to each round hole, and obtaining the round detection result of the part picture.

[0058] In a preferred embodiment, the semantic segmentation network model is a BiSeNet network model.

[0059] The picture set D obtained through step S02 is input into two branches, one branch is a space branch S, and the other is a semantic branch C. The picture set S of the space branch passes through three consecutive convolution modules Φ, each convolution module is composed of a convolution kernel with a step of 2, a batch normalization layer (Batch Normalization) and a ReLU activation function. After passing through the three convolution modules, a new feature map Sm with a feature map size of 1 / 8 of the original picture set is obtained. Figure 1 The semantic branch passes through three downsampling operations and is sequentially input into two attention refinement (Attention Refinement Module) modules α for data fusion. One attention refinement module includes a global pooling layer, a 1x1 convolution kernel and a batch normalization layer. In the selection of the activation function at the end of the module, the present application uses a Leaky ReLU activation function instead of the original Sigmoid function to alleviate the gradient disappearance phenomenon and speed up the loss function convergence. Finally, the feature map Sm of the space branch and the feature map Cm of the semantic branch are input into a feature fusion module (Feature Fusion Module) to finally obtain the contour segmentation result heat map H of the circle. Wherein:

[0060]

[0061] Wherein, is a bilinear interpolation operation, Φ represents the convolution module of the space branch, and α represents the attention refinement module.

[0062] The contour segmentation result heat map H of the circle is obtained, that is, the detection of the round hole in the part picture is completed.

[0063] Preferably, in the training of the semantic segmentation network model, after obtaining the round hole detection picture set D, the picture set size is first uniformly processed to 256x256, and pictures smaller than the size are filled with a white background. Then, the Labelme image labeling software tool is used to label the contour of the round hole in the picture. The semantic segmentation labeling uses a line segment connected by multiple points to enclose different types of objects, respectively, to generate a json file with pixel information of about 600 class labels.

[0064] Using the above json file with the pixel information with category labels, the training of the semantic segmentation network model can be completed.

[0065] In actual application scenarios, the existing deep learning detection task cannot directly calculate the parameters of the circle, and other measurement methods need to be used to calculate the parameters of the circle separately after the circle is detected, and the application designs an end-to-end circle detection and parameter calculation system, which can directly obtain the parameters of the circle by inputting the part image without additional operations.

[0066] In a more preferred embodiment, the part circle detection method provided by the application further comprises the steps of:

[0067] S04: For each circle hole corresponding to the contour segmentation result heat map, a random sample consensus method is used to obtain the circle parameters corresponding to the circle hole.

[0068] Specifically, the following sub-steps are included:

[0069] S11: For each corresponding contour segmentation result heat map, three points are randomly selected from the point set to calculate a circle model.

[0070] S12: All points in the point set are substituted into the model to calculate the number of inliers that meet the threshold requirement.

[0071] S13: Compare the number of inliers in the current model with the number of inliers in the current best model, and record the model parameters and the number of inliers with the maximum number of inliers.

[0072] S14: Repeat steps S11-S13 until the number of iterations reaches a specified value or the number of inliers in the current model is greater than a set threshold value.

[0073]

[0074] P=1-(1-t n ) k

[0075]

[0076] Wherein, t is the proportion of inliers in the first iteration model, Nin is the number of inliers, Nout is the number of outliers, n is the number of points used in each iteration, k is the number of iterations, and P is the probability of sampling n inliers in k iterations.

[0077] The application embodiment further provides a part circle detection device, as shown in the figure, the part circle detection device 400 includes: Figure 3 As shown in the figure, the part circle detection device 400 includes:

[0078] The part picture acquisition module 401 is configured to acquire a part picture to be detected, wherein the part picture contains a plurality of round holes;

[0079] The round hole result atlas acquisition module 402 is configured to input the part picture into a trained target detection network model to obtain a round hole result atlas corresponding to the part, wherein the round hole result atlas includes a single round hole picture corresponding to all round holes in the part picture.

[0080] The detection result acquisition module 403 is configured to input the round hole result atlas into a trained semantic segmentation network model to calculate a contour segmentation result heat map corresponding to each round hole, and obtain a round detection result of the part picture.

[0081] Preferably, the system further comprises a round parameter calculation module configured to, for each contour segmentation result heat map corresponding to a round hole, obtain a round parameter corresponding to the round hole by using a random sample consensus method.

[0082] Preferably, the target detection network model is a YOLOv5 model.

[0083] Preferably, the prior box size of the YOLOv5 model is [10:13, 26:30, 33:29], [55:61, 62:54, 108:119], and [116:110, 190:198, 338:326].

[0084] Preferably, the semantic segmentation network model is a BiSeNet network model.

[0085] Preferably, the activation function of the BiSeNet network model is a Leaky ReLU activation function.

[0086] Preferably, the round parameter calculation module performs the following steps:

[0087] S11: For each corresponding contour segmentation result heat map, randomly select three points from the point set to calculate a model of a circle;

[0088] S12: Substitute all points in the point set into the model to calculate the number of inliers satisfying a threshold requirement;

[0089] S13: Compare the number of inliers of the current model with the number of inliers of the current best model, and record the model parameters and the number of inliers of the maximum number of inliers;

[0090] S14: Repeat steps S11-S13 until the number of iterations reaches a specified value or the number of inliers of the current model is greater than a set threshold value. Wherein:

[0091]

[0092] P = 1 - (1 - t n ) k

[0093]

[0094] wherein t is the ratio of inliers in a model of one iteration, Nin is the number of inliers, Nout is the number of outliers, n is the number of points used in each iteration, k is the number of iterations, and P is the probability that n inliers are sampled in k iterations.

[0095] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the part of the description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units. It can be clearly understood by those skilled in the art that, for the convenience and brevity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiments, and will not be described here.

[0096] The embodiments of the present application further provide an electronic device, comprising:

[0097] at least one memory and at least one processor;

[0098] the memory is configured to store one or more programs;

[0099] When the one or more programs are executed by the at least one processor, the at least one processor implements the steps of the part circle detection method as described above.

[0100] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the part of the description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the present disclosure. Those skilled in the art can understand and implement without creative labor.

[0101] The embodiments of the present application further provide a computer readable storage medium,

[0102] The computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the part circle detection method as described above.

[0103] Computer-usable storage media include permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0104] The application provides a part circle detection method and device, electronic equipment and storage medium. The part circle is recognized by a target detection method, and the contour information of the part circle is obtained by a semantic segmentation method. Finally, the parameters of the part circle are estimated by a random sample consensus method. In view of the problem that the existing part circle detection method based on deep learning lacks an open source data set, the open source data set of the part circle is constructed, and the development of the part circle detection method is promoted. In view of the problem that the existing circle detection method has a narrow application scene and low precision, the circle detection and parameter calculation for a complex part scene are realized, and the application scene of the circle detection method is widened.

[0105] The existing detection technology can usually only recognize the overall contour of an object and cannot detect specific fine features of the object. In view of this problem, the application can detect the relatively fine features such as the part circle, and lays a foundation for intelligent quality detection of industrial production.

[0106] The existing deep learning detection task cannot directly calculate the parameters of the part circle, and needs to separately calculate the parameters of the part circle by using other measurement methods after the part circle is detected. In view of this, the application designs an end-to-end circle detection and parameter calculation system, and the parameters of the part circle can be directly obtained by inputting a part image without additional operations. The above-described embodiments only express several implementation manners of the application, and the description is relatively specific and detailed, but the application scope should not be limited in this way. It should be pointed out that, for ordinary skilled persons in the art, without departing from the concept of the application, a plurality of modifications and improvements can be made, and these all belong to the protection scope of the application.

Claims

1. A method for detecting the roundness of a part, characterized in that, Includes the following steps: Obtain an image of the part to be inspected, wherein the image contains several circular holes; The part image is input into the trained target detection network model to obtain the circular hole result image set corresponding to the part. The circular hole result image set includes the single circular hole image corresponding to all the circular holes in the part image. The set of circular hole results is input into the trained semantic segmentation network model to calculate the heat map of the contour segmentation result corresponding to each circular hole. For the heat map of the contour segmentation result corresponding to each circular hole, the random sampling consensus method is used to obtain the circle parameters corresponding to the circular hole, and the circle detection result of the part image is obtained. The random sampling consensus method includes: S11: For each corresponding contour segmentation result heatmap, three points are randomly selected from the point set to calculate a circle model; S12: Substitute all points in the point set into the model to calculate the number of interior points that meet the threshold requirement; let t be the proportion of interior points in one iteration of the model, and N in Let N be the number of interior points. out For the number of outgoing points, ; S13: Compare the number of interior points in the current model with the number of interior points in the current best model, and record the model parameters and the number of interior points of the model with the largest number of interior points; the best model is the model with the largest number of interior points. S14: Repeat steps S11-S13 until the number of iterations reaches a specified value or the number of points in the current model exceeds a set threshold; let n be the number of points used in each iteration, P be the probability of sampling n points in k iterations, and k be the number of iterations. The conclusion is .

2. The method for detecting the roundness of a part according to claim 1, characterized in that: The target detection network model is the YOLOv5 model.

3. The method for detecting the roundness of a part according to claim 2, characterized in that: The prior bounding box dimensions of YOLOv5 are [10:13, 26:30, 33:29], [55:61, 62:54, 108:119], and [116:110, 190:198, 338:326].

4. The method for detecting the roundness of a part according to claim 1, characterized in that: The semantic segmentation network model is the BiSeNet network model.

5. The method for detecting the roundness of a part according to claim 4, characterized in that: The activation function of the BiSeNet network model is the Leaky ReLU activation function.

6. A part roundness detection device, characterized in that, include: A part image acquisition module is used to acquire images of the parts to be inspected, wherein the part images contain several circular holes; The circular hole result image set acquisition module is used to input the part image into the trained target detection network model to obtain the circular hole result image set corresponding to the part. The circular hole result image set includes individual circular hole images corresponding to all circular holes in the part image. The detection result acquisition module is used to input the circular hole result image set into the trained semantic segmentation network model, calculate the contour segmentation result heat map corresponding to each circular hole, and use the random sampling consensus method to obtain the circle parameters corresponding to the circular hole for each circular hole, thereby obtaining the circle detection result of the part image. The random sampling consensus method includes: S11: For each corresponding contour segmentation result heatmap, three points are randomly selected from the point set to calculate a circle model; S12: Substitute all points in the point set into the model to calculate the number of interior points that meet the threshold requirement; let t be the proportion of interior points in one iteration of the model, and N in Let N be the number of interior points. out For the number of outgoing points, ; S13: Compare the number of interior points in the current model with the number of interior points in the current best model, and record the model parameters and the number of interior points of the model with the largest number of interior points; the best model is the model with the largest number of interior points. S14: Repeat steps S11-S13 until the number of iterations reaches a specified value or the number of points in the current model exceeds a set threshold; let n be the number of points used in each iteration, P be the probability of sampling n points in k iterations, and k be the number of iterations. The conclusion is .

7. An electronic device, characterized in that, include: At least one memory and at least one processor; The memory is used to store one or more programs; When the one or more programs are executed by the at least one processor, the at least one processor implements the steps of a part circle detection method as described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a part roundness detection method as described in any one of claims 1-5.

Images