Industrial control valve-oriented deep learning target detection method and system
Through the design of the POD-YOLO model, the accuracy and robustness of the existing part target detection method in complex scenarios of industrial control valve parts is solved, and high accuracy and rapid detection are achieved, which is suitable for complex parts inspection.
Patent Information
- Application Number
- CN202411181213.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-12-20
AI Technical Summary
Existing part object detection methods are less accurate when dealing with complex industrial control valve parts, especially poor performance on part-intensive, large size differences and lack of data sets, and traditional methods are susceptible to image quality and lighting. It is difficult to deal with large-scale and diverse scenarios.
The POD-YOLO model was designed, using the improved YOLOv5 backbone and head, and the CDCM module and MSC-FPN layer were introduced to optimize feature extraction and detection heads, enhance the receptive field and robustness of the model, adapt to feature extraction of small-sized parts, and pass Multi-scale context information feature pyramid improves detection performance.
The POD-YOLO model has an overall mAP of 93.00%, and the detection speed is comparable to that of YOLOv5s. It has excellent detection performance and robustness, which significantly improves the accuracy and generalization ability of control valve parts. It is suitable for complex scenarios and diverse parts. Test.
Smart Images
Figure CN119169350A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of control valve technology, and in particular to a deep learning target detection method and system for industrial control valves. Background Art
[0002] As an important branch of computer vision, object detection is widely used in many fields such as robot sorting, video surveillance, industrial production, etc., and has important practical significance.
[0003] As a specific task in the field of industrial target detection, part target detection focuses on automatically identifying and locating various parts in industrial production. Control valve part target detection plays a very important role in process industry and dispersion industry.
[0004] Part target detection methods based on machine learning are usually divided into two stages: target extraction and classification. In the target extraction stage, there are many methods that can be used to extract targets in images, such as using edge detection algorithms to obtain edge information of objects, and then extracting target areas based on edge information. In addition, feature-based methods can also be used, such as using SIFT features, HOG features, etc. to obtain feature information of target objects, and then extracting target areas. In the classification stage, once the target area is obtained and the corresponding feature information is extracted, the extracted target can be classified by a classifier. Although this traditional part target detection method is relatively simple, it requires manual design of feature extraction methods and classifiers, and is easily affected by factors such as image quality and lighting, so the accuracy is relatively low. At the same time, it is not suitable for processing large-scale data and diverse scenes. There are some limitations when dealing with complex part shapes, background changes, scale changes, and occlusions.
[0005] Methods based on deep learning have achieved remarkable results in target detection tasks. In parts detection, deep learning models can learn by training a large number of part images to understand and identify the features and morphology of different parts. Compared with traditional rule-based or manual feature extraction methods, part detection based on deep learning can learn and extract part features more automatically, thereby improving the accuracy and applicability of detection, and can handle more complex scenes and changes. Typical single-stage target detection algorithms include YOLOv3, YOLOv5, YOLOv7, YOLOv8, SSD, CenterNet, Faster-RCNN, etc.; However, these methods need to be improved in terms of accuracy, detection speed, and other indicators when parts are densely populated and occluded, when parts have large size differences, and when there is a lack of part data sets. Summary of the invention
[0006] Aiming at the shortcomings of the existing methods, the present invention designs a CDCM module based on the C3 module of YOLOv5, and proposes MSC-FPN according to the characteristics of many small-sized parts in the control valve dataset of actual industrial scenarios; the prediction head and backbone of YOLOv5 are improved according to the small-sized parts and large-sized parts of the control valve parts.
[0007] The technical solution adopted by the present invention is: a deep learning target detection method for industrial control valves comprises the following steps:
[0008] Step 1: Collect image data of control valve parts, and annotate and enhance the images;
[0009] In a preferred embodiment of the present invention, the parts of the control valve include: a guide sleeve, a sleeve, a columnar valve stem, a rod-shaped valve stem, an upper sleeve, an upper valve cover, a valve body, a valve seat and a valve core.
[0010] Step 2: Build a POD-YOLO model. The backbone of the POD-YOLO model includes: a first CBS convolution module, a second CBS convolution module, a third C3 module, a fourth CBS convolution module, a fifth C3 module, a sixth CBS convolution module, a seventh C3 module, and an eighth SPFF module connected in sequence;
[0011] Step 3: Use the images of control valve parts to train the POD-YOLO model and perform target recognition on the parts of the control valve.
[0012] In a preferred embodiment of the present invention, the FPN layer of the POD-YOLO model Neck includes a ninth CBS convolution module, a tenth upsampling module, an eleventh Concat module, a twelfth CDCM module, a thirteenth CBS convolution module, a fourteenth upsampling module, a fifteenth Concat module, a sixteenth CDCM module, a seventeenth CBS convolution module, an eighteenth C3 module, a nineteenth CBS module, a twentieth C3 module, a twenty-first SPPF module, a twenty-second CBS module, a twenty-third upsampling module, a twenty-fourth Concat module, a twenty-fifth C3 module, a twenty-sixth CBS module, a twenty-seventh upsampling module, a twenty-eighth Concat module and a twenty-ninth C3 module connected in sequence, the ninth CBS module is connected to the eighth SPFF module, the eleventh Concat module is connected to the fifth C3 module, the fifteenth Concat module is connected to the third C3 module, the sixteenth CDCM module is also connected to the twenty-eighth Concat module, and the eighteenth C3 module is also connected to the twenty-fourth Concat module.
[0013] In a preferred embodiment of the present invention, the right branch of the CDCM module includes a series of dilated convolutions with dilation rates of 1, 2, 5, and 1, and the right branch is residually connected to the CBS module of the left branch and then the CBS module is connected in series.
[0014] In a preferred embodiment of the present invention, the thirty-first and thirty-fourth Concat modules of the PAN layer of the POD-YOLO model Neck are respectively connected to the twenty-sixth and twenty-second CBS modules of the FPN layer, and the thirtieth CBS module of the PAN layer is connected to the twenty-ninth C3 module of the FPN layer.
[0015] In a preferred embodiment of the present invention, the sizes of the three output feature maps of the Head detection head of the POD-YOLO model are 160×160, 80×80 and 40×40 respectively.
[0016] A preferred embodiment of the present invention is a deep learning target detection system for industrial control valves, comprising: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement a deep learning target detection method for industrial control valves.
[0017] A preferred embodiment of the present invention is a computer-readable medium storing computer program code, which, when executed by a processor, implements a deep learning target detection method for industrial control valves.
[0018] Beneficial effects of the present invention:
[0019] 1. Through the analysis of the overall mAP, the accuracy of the POD-YOLO model of the present invention is 93.00%; compared with other models, the POD-YOLO model has obvious advantages in the overall detection effect, which fully proves the effectiveness and superiority of the model;
[0020] 2. Comparing the detection speed FPS, the POD-YOLO model is 50, which is equivalent to YOLOv5s, but the accuracy is significantly better than YOLOv5s, which shows that the POD-YOLO model has excellent detection performance while ensuring high detection speed;
[0021] 3. From the accuracy analysis of various types of parts, POD-YOLO has the best overall performance and good generalization ability and robustness, indicating that the POD-YOLO model has superior performance and effectiveness in the control valve parts target detection task. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 This is the POD-YOLO network structure diagram;
[0023] Figure 2 This is a comparison diagram of the C3 and CDCM structures;
[0024] Figure 3 This is a schematic diagram of the working principle of the multi-scale context information feature pyramid;
[0025] Figure 4 It is a detailed map of the multi-scale context information feature pyramid;
[0026] Figure 5 This is a comparison chart of the backbone and head of POD-YOLO and YOLOv5s. DETAILED DESCRIPTION
[0027] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.
[0028] like Figure 1 As shown, the deep learning target detection method for industrial control valves includes the following steps:
[0029] The POD-YOLO model of the present invention includes four parts: Input, Backbone, Neck and Head. Among them, the Input is the same as the original YOLOv5 model, and data enhancement is performed on the input image.
[0030] To solve the problems of unbalanced part samples and difficulty in capturing the features of small-sized parts, the present invention improves the Backbone and the Head respectively; first, the size of the Backbone feature information output is changed to better adapt to the feature extraction of small-sized parts; second, in the Head detection part, a smaller target detection head is introduced to capture the features of small-sized parts, and the function of the original large target detection head is replaced by the medium target detection head, leaving only the smaller detection head, the small detection head and the medium detection head; these two improvements are helpful to improve the accuracy and robustness of target detection, thereby better solving the problems of unbalanced part samples and difficulty in capturing the features of small-sized parts.
[0031] Among them, the Backbone optimization includes: in view of the small size of control valve parts and the large number of targets, the feature map output of Backbone is adjusted and optimized. The feature map sizes of the original YOLOv5 output are 80×80, 40×40 and 20×20. Due to the existence of very small targets, it is necessary to output the 160×160 feature map containing more information of smaller targets; therefore, the feature map size of the Backbone output is changed to 160×160, 80×80 and 40×40; although the feature map of 20×20 is reduced, the feature map of 40×40 can ensure the coverage of 20×20 feature information; in addition, the receptive field is expanded by introducing the Cross Stage Dilated Convolution Module (CDCM), which makes up for the receptive field of 20×20 to a certain extent. Figure 5 (a) shows Backbone before and after optimization; Backbone extracts important features from the image through the optimized convolutional neural network, and inputs feature maps of sizes 160×160, 80×80, and 40×40 to the Neck part for the next step of feature fusion.
[0032] The optimization of the Head detection head includes: the original YOLOv5 uses 80×80, 40×80 and 20×20 feature maps to detect small targets, medium targets and large targets respectively; the control valve parts have large size differences, and there are more small parts, which occupy fewer pixels on the image; the 20×20 feature map used to predict small targets is the result of 8x downsampling, and the 8x downsampling grid corresponds to the 8×8 area on the input image, which is not very friendly to smaller targets with a size less than 8×8, and it is easy to cause the feature information of smaller targets to be lost, resulting in missed detection and wrong detection of smaller targets; for this reason, the Head of YOLOv5 is improved, a smaller target detection head is introduced, and a 160×160 feature map is used to predict smaller targets; an 80×80 feature map is used to predict small and medium targets; a 40×40 feature map is used to predict large targets. The number of parameters of the optimized Head detection head is significantly reduced, which improves the detection speed. Figure 5 (b) shows the Head detection head before and after optimization. The Head detection head generates anchor boxes of different sizes for the feature map output by the feature fusion stage of the Neck layer, and then uses NMS (Non-Maximum Suppression) to remove redundant prediction boxes and generate the category of the target.
[0033] In addition, the Neck layer uses the designed CDCM and multi-scale context information feature pyramid (MSC-FPN) to generate visual features of multi-scale context information using the multi-level features in Backbone, and then guides the generation of semantic features with multi-scale context information, further increasing the receptive field of the network and enhancing the ability to express features of small and large parts.
[0034] Since the sizes of objects in the same image vary greatly, there will be a large receptive field error. Large objects may occupy most of the image area, causing small objects to be ignored or missed.
[0035] YOLOv5 uses the C3 module without residual connection in the Neck layer, such as Figure 2 As shown in a, the receptive field is expanded by a BottleNeck structure composed of 1x1 convolution and 3x3 convolution in series, but the receptive field after the C3 module is expanded is much smaller than the difference in part size, which is insufficient to support the size difference of part size; if the number of BottleNecks is further increased, the complexity of the model will increase and information redundancy will be caused; therefore, the present invention proposes a double-branch cross-stage dilated convolution module based on the idea of dilated convolution and cross-stage hierarchical network; Figure 2 As shown in (b), CDCM consists of two branch structures, the left and right branches have the same number of channels; the right branch expands the receptive field of the model through multiple serial dilated convolutions to capture high-level features of different scales; the left branch is composed of CBS modules, which are mainly responsible for capturing low-level features; by splicing the left branch to the output result of the dilated convolution branch of the right branch, the difference between different receptive fields is compensated, which plays a role in enhancing features, thereby improving the learning ability of the network; the CDCM module is introduced at the position of the original C3 module after the Backbone performs feature fusion in the Neck layer, which fully contains multi-scale contextual information; after the introduction of the CDCM module, on the one hand, the receptive field is increased, and on the other hand, it avoids information redundancy through sparse sampling. In the Neck layer, multi-scale contextual visual information and semantic information can be effectively combined to help the model make predictions. The specific process of the CDCM module is as follows: Figure 2 As shown in (b), the dilated convolution branches on the right side of the cross-stage dilated convolution module are connected in series according to the dilation rates 1, 2, 5, and 1; where f Conv,d＝1 Equivalent to 1×1 convolution; f Conv,d＝2 is the dilated convolution with a dilation factor of 2; f Conv,d＝5 It is an expanded convolution with an expansion factor of 5. Two 1×1 convolutions are used for dimensionality increase and dimensionality decrease respectively to reduce the number of parameters and keep the dimension unchanged. The feature map F1 output by the right expanded convolution branch is expressed as:
[0036] F1=fConv,d＝1 (f Conv,d＝5 (f Conv,d =2(f Conv,d＝1 (I)))) 1)
[0037] The left side is the CBS module branch. Through the processing of the CBS module, a more abstract and representative feature representation can be obtained; where: f Conv is a 1×1 convolution; f BN is the BN layer; f Silu is the Silu activation function, and the expression of the feature map F2 output by the left CBS module branch is:
[0038] F2=f Silu (f BN (f Conv (x))) (2)
[0039] After the feature maps of the left and right branches are processed, the low-level features and high-level features will be fused at Concat. The number of channels after fusion is the same as before entering the cross-stage dilated convolution module. The expression of the fused feature map F3 is:
[0040]
[0041] The fused feature map F3 passes through a CBS module again, and the expression of the final output feature map F is:
[0042] F=f Silu (f BN (f Conv (F3))) (4)
[0043] By combining the idea of expanding the receptive field and gradient splitting, the model can process targets of different scales at the same time, thereby improving the robustness of detection, increasing the receptive field without increasing the number of parameters, increasing the richness of contextual information, and generating features with multi-scale contextual information.
[0044] Construct a multi-scale contextual information feature pyramid The Neck layer of YOLOv5 extracts shallow visual features from the Backbone and connects them with deeper semantic features. The FPN structure of the Neck layer increases the scale of the feature map by upsampling, which is convenient for fusing the feature map from the Backbone and realizing upward fusion of features. The PAN structure of the Neck layer continues to downsample, on the one hand to obtain feature maps of different scales, and on the other hand to achieve better fusion of shallow graphic features with deep semantic features. However, in the working environment, large part targets may occupy a larger image area, while small part targets may only occupy a small area. Due to the large difference in part sizes, the FPN in the Neck layer may find it difficult to capture all target information of different sizes, which may make it difficult to detect or identify small part targets, and multi-scale context information is needed to guide feature extraction. To this end, the FPN structure in the original Neck layer is improved, and the MSC-FPN layer is proposed to guide network feature extraction and fusion with visual information with multi-scale context and semantic information with multi-scale context, thereby improving the model's ability to capture large-scale information and small-scale information.
[0045] like Figure 3 As shown in the figure, the specific role of the MSC-FPN layer is to use the visual feature P3′ with multi-scale context information generated by the CDCM module to guide the generation of the semantic feature P4″ with multi-scale context information. The semantic feature captures both the feature information at a large scale and the feature information at a small scale. The visual feature P3′ with multi-scale context information has more guidance from the multi-scale context information than the visual feature extracted by the original FPN structure. The semantic feature P4″ with multi-scale context information guides the visual feature P3″ with multi-scale context information to generate P2″ with the ability to have a common receptive field for large and small targets, and then uses the precise positioning information in the lower layer through the PAN structure to enhance the path, thereby further enhancing the positioning capability of the entire feature extraction network.
[0046] like Figure 4As shown in the figure, the specific process of the MSC-FPN layer is as follows: in the first upsampling process, the low-level features and high-level features of different layers of Backbone input are accepted to generate features with multi-scale context information; the features with context information are downsampled again to fully extract the key features of different levels, and SPPF is used for pooling in the final stage of downsampling; then the upsampling part is entered again, and the feature information generated by the downsampling in the previous stage is used to guide the generation of feature information with different receptive fields; the feature maps generated by the CBS module in the second upsampling stage and the final C3 module will enter the PAN structure of the final downsampling part. The PAN structure is opposite to the feature transmission path of the upsampling part, which can shorten the information path and make full use of the precise positioning information in the low-level feature map, thereby improving the positioning ability of the model.
[0047] Experimental process:
[0048] Experiment 1: Verify the performance of different target detection models on the PD4CV2023 dataset;
[0049] The first group of experiments is a comparative experiment on the part detection performance of different target detection models on the PD4CV2023 dataset. The experiment compares the performance of the POD-YOLO model of the present invention with the two-stage Faster-RCNN, the single-stage SDD, YOLOv3, YOLOv5s and YOLOv7l, the key point-based CenterNet, the Transformer-based DETR, the anchor-free YOLOXs and YOLOv8s. By comparing the performance with the existing methods, the performance of the POD-YOLO model in the control valve part target detection task can be evaluated.
[0050] Table 1 Comparison of results of different target detection algorithms on PD4CV2023 dataset (%)
[0051]
[0052] 1. Through the analysis of the overall mAP, the POD-YOLO model of the present invention has achieved the highest level of accuracy, with an accuracy rate of 93.00%; compared with other models, the POD-YOLO model has obvious advantages in the overall detection effect, which fully proves the effectiveness and superiority of the model.
[0053] 2. By comparing the detection speed FPS, we can find that the fastest detection speed is YOLOv5s, whose FPS value is quite high, reaching 51; however, the mAP value of YOLOv5s is only 89.10%, while at the same speed, the mAP value of POD-YOLO has achieved 93.00%, which fully demonstrates that the POD-YOLO model has excellent detection performance while ensuring high detection speed.
[0054] 3. From the accuracy of each type of parts, although the number of POD-YOLO achieving the highest accuracy in each category is relatively small, in terms of overall performance, POD-YOLO still achieved competitive results, demonstrating good generalization ability and robustness, indicating that the POD-YOLO model has superior performance and effectiveness in the control valve parts target detection task.
[0055] Experiment 2: Detection performance test at different density levels
[0056] In order to verify the effect of the POD-YOLO model on improving the problem of dense parts, Experiment 2 roughly divides the test set of the PD4CV2023 dataset into a small density group according to the number of parts in the image being less than 40, a medium density group according to the number of parts being greater than 40 and less than 50, and a large density group according to the number of parts being greater than 50; in each group of data, the POD-YOLO model is used to conduct comparative experiments with YOLOv5s, and the detection performance of the two models on three groups of images is compared to verify the detection performance under different densities of parts; in the three groups of data, the sizes of parts with small density differ slightly, the sizes of parts with medium density do not differ much, and the sizes of parts with large density differ greatly.
[0057] The first group of experiments is to detect YOLOv5s and POD-YOLO on images with low part density. The smaller the part density, the greater the difference in the overall size of the parts, and generally there will be no confusion of parts. As shown in Table 2, it can be seen that POD-YOLO has improved the accuracy of small, medium and large parts in the test set with low part density, especially the medium-sized rod-shaped valve stem and valve core, and the ability to distinguish these two types of similar parts has been improved.
[0058] Table 2 Comparison of test results of parts with low density (%)
[0059]
[0060] The second group of experiments is YOLOv5s and POD-YOLO detection on images with medium part density. The part density is medium and the overall size of the parts is not very different. As can be seen from Table 3, POD-YOLO has significantly improved the accuracy of medium-sized parts in the test set with medium part density, and has a better detection effect on medium-sized parts with medium part size differences.
[0061] Table 3 Comparison of test results of medium-density parts (%)
[0062]
[0063] The third group of experiments is to detect YOLOv5s and POD-YOLO on images with high part density. The greater the part density, the smaller the overall difference in the size of the parts, and part confusion will occur. As can be seen from Table 4, POD-YOLO has improved the accuracy of small, medium and large parts in the test set with high part density, and can achieve better results than YOLOv5s for images with high part size density.
[0064] Table 4 Comparison of test results of parts with high density (%)
[0065]
[0066] Note: The values in bold are the optimal values, and the values in underline are the suboptimal values.
[0067] Overall, POD-YOLO achieved the best results in two sets of experiments: performance testing of different target detection models on the PD4CV2023 dataset and detection performance testing at different densities. This fully demonstrates the effectiveness of the POD-YOLO model and proves that POD-YOLO is suitable for target detection of industrial control valve parts.
[0068] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.
Claims
1. A deep learning target detection method for industrial control valves, characterized in that: The following steps are involved: Step 1: Collect image data of control valve parts, and annotate and enhance the images; Step 2: Build a POD-YOLO model. The backbone of the POD-YOLO model includes: a first CBS convolution module, a second CBS convolution module, a third C3 module, a fourth CBS convolution module, a fifth C3 module, a sixth CBS convolution module, a seventh C3 module, and an eighth SPFF module connected in sequence; Step 3: Use the images of control valve parts to train the POD-YOLO model and perform target recognition on the parts of the control valve.
2. The deep learning target detection method for industrial control valves according to claim 1 is characterized in that: The FPN layer of the POD-YOLO model Neck includes a ninth CBS convolution module, a tenth upsampling module, an eleventh Concat module, a twelfth CDCM module, a thirteenth CBS convolution module, a fourteenth upsampling module, a fifteenth Concat module, a sixteenth CDCM module, a seventeenth CBS convolution module, an eighteenth C3 module, a nineteenth CBS module, a twentieth C3 module, a twenty-first SPPF module, a twenty-second CBS module, a twenty-third upsampling module, a twenty-fourth Concat module, a twenty-fifth C3 module, a twenty-sixth CBS module, a twenty-seventh upsampling module, a twenty-eighth Concat module and a twenty-ninth C3 module, which are connected in sequence. The ninth CBS module is connected to the eighth SPFF module, the eleventh Concat module is connected to the fifth C3 module, the fifteenth Concat module is connected to the third C3 module, the sixteenth CDCM module is also connected to the twenty-eighth Concat module, and the eighteenth C3 module is also connected to the twenty-fourth Concat module.
3. The deep learning target detection method for industrial control valves according to claim 2 is characterized in that: The right branch of the CDCM module includes a series of dilated convolutions with dilation rates of 1, 2, 5, and 1. The right branch is residually connected to the CBS module of the left branch and then the CBS module is connected in series.
4. The deep learning target detection method for industrial control valves according to claim 3 is characterized in that: The 31st and 34th Concat modules of the PAN layer of the POD-YOLO model Neck are connected to the 26th and 22nd CBS modules respectively, and the 30th CBS module of the PAN layer is connected to the 29th C3 module.
5. The deep learning target detection method for industrial control valves according to claim 4 is characterized in that: The three output feature map sizes of the Head detection head of the POD-YOLO model are 160×160, 80×80, and 40×40 respectively.
6. The deep learning target detection method for industrial control valves according to claim 1 is characterized in that: The parts of the control valve include: guide sleeve, sleeve, cylindrical valve stem, rod-shaped valve stem, upper sleeve, upper valve cover, valve body, valve seat and valve core.
7. A deep learning target detection system for industrial control valves, characterized by: include: a memory for storing instructions executable by a processor; A processor, configured to execute instructions to implement the deep learning target detection method for industrial control valves as described in any one of claims 1 to 6.
8. A computer readable medium storing a computer program code, characterized in that: When the computer program code is executed by a processor, the computer program code implements the deep learning target detection method for industrial control valves as described in any one of claims 1 to 6.