Industrial application-oriented real-time target detection model and detection method
By using structural reparameterization technology and self-distillation and pruning methods in the object detection model, the network structure is optimized, and the problem of insufficient deployment effect and small and medium data set detection effect is solved, and efficient deployment on embedded devices is achieved. Improved detection accuracy.
Patent Information
- Application Number
- CN202411197037.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-06
AI Technical Summary
The existing object detection model has shortcomings in deployment effects and detection effects on small and medium-type datasets, and it is difficult to deploy on embedded devices due to the computing and storage resource limitations of the embedded platform.
A real-time object detection model for industrial applications was designed, using Backbone module for feature extraction, Neck module for multi-scale feature fusion, Head module for detection and classification, using structural reparameterization technology to optimize the network structure, and self-distillation and shearing Branch and quantitative experiments improve the deployment effect and generalization performance of the model.
It realizes efficient deployment on embedded devices, improves detection accuracy and inference speed, and performs better than other models on small and medium-sized data sets, and is suitable for real-time object detection for industrial applications.
Smart Images

Figure CN119091153A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of target detection, and in particular relates to a real-time target detection model and a detection method for industrial applications. Background Art
[0002] The rapid development of deep learning has led to major breakthroughs in all aspects of computer vision, especially in object detection. Object detection is an important research topic in computer vision and an indispensable component of computer vision systems. Its purpose is to identify object categories and locate object positions. It has a wide range of applications in autonomous driving, industrial inspection, robotics, medical image processing, target tracking and other fields.
[0003] The commonly used method in the field of target detection is the YOLO series in CNN. In recent years, Transformer has made great achievements in the field of computer vision with its advantages of good parallelism, strong global modeling ability, and strong robustness. However, the Transformer network model is relatively complex and has high training costs. To achieve good detection results, it usually depends on pre-training its backbone network on the ImageNet dataset. In addition, Transformer is difficult to deploy, and hardware that is friendly to Transformer deployment is still in its infancy. The recently proposed real-time target detector RT DETR does not have corresponding pre-trained weights when used in specific fields. The DETR series of target detectors are difficult to apply to new fields, so the YOLO series is still the most widely used real-time target detector.
[0004] YOLOv1 is an end-to-end detection model. YOLOv1 treats the target detection problem as a regression problem. This model greatly improves the detection speed, but the accuracy is relatively low. YOLOv2 introduces nine methods such as batch normalization and high-resolution classifiers on the basis of YOLOv1, and improves the detection accuracy of YOLOv1 by 15.2% on the VOC2007 dataset. YOLOv3 was proposed by Redmon in 2018. It introduces the FPN structure on the basis of YOLOv2 to better detect targets of different sizes. YOLOv1-v3 builds the initial YOLO network, divides the network structure into three parts: trunk-neck-head, and predicts targets of different sizes through multi-scale branches. YOLOv4 introduces the bottom-up PAN structure, CIOU bounding box loss function, Mosaic and other technologies on the basis of YOLOv3, and takes a step further in detection accuracy. YOLOv5 integrates various SOTA methods, combining accuracy, speed and flexibility. Subsequently, YOLO X introduced Multi positive, Anchor-free and decoupling Head, creating a new paradigm for YOLO model design. YOLOv6 introduced the reparameterization method into the YOLO series of models for the first time, and proposed EfficientRep Backbone and Rep-PAN Neck. YOLOv7 focused on analyzing the impact of gradient paths on model performance, and proposed the E-ELAN structure to enhance model capabilities without destroying the original gradient path. PP-YOLOE introduced CSPRepResStage, ET-head and dynamic label assignment algorithm TAL to achieve friendly deployment. RTMDet is based on YOLOX, adopts a new training strategy, introduces large-core deep convolution, and designs a new soft label assignment, which further improves the detection effect. YOLOv8 absorbs the advantages of previous YOLO models and integrates them together to achieve excellent detection results. Gold-YOLO discovered the information fusion problem of the model, proposed the GD mechanism, improved the multi-scale feature fusion capability, and achieved SOTA in the YOLO network. However, most of the above excellent detection models focus on improving the detection accuracy on large datasets such as COCO, while ignoring the deployment effect and the detection effect on other small and medium-sized datasets.
[0005] In addition, traditional target detection models are usually deployed on cloud servers, which have great demands on computing and storage resources. When intelligent analysis of surveillance videos is required, the surveillance videos must be transmitted to the cloud server through the network. Then, the surveillance videos are analyzed by the AI model in the cloud server, and the video analysis results are returned through the network. Due to the limitation of network bandwidth, the entire process of cloud computing has serious transmission delays. At the same time, a large number of surveillance video transmissions will cause serious network congestion. Using edge computing to disperse intelligent computing close to the data source can avoid transmission delays and network congestion, but edge computing usually needs to process a large amount of data, so its power consumption is high, and the equipment maintenance is difficult and the scalability is poor. Deploying target detection models on embedded platforms can not only avoid the problems caused by cloud computing, but also control equipment or alarm devices according to real-time analysis results of the video. At the same time, embedded devices have low power consumption and good maintainability. However, due to the limitations of computing and storage resources of embedded platforms, traditional detection models are difficult to deploy on embedded devices. Summary of the invention
[0006] The purpose of the embodiments of the present invention is to provide a real-time target detection model and detection method for industrial applications, so as to solve the problem that the existing detection model ignores the deployment effect and the detection effect on other small and medium-sized data sets, and is difficult to be deployed on embedded devices due to the limitations of the computing and storage resources of the embedded platform.
[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is a real-time target detection model for industrial applications, comprising:
[0008] Backbone module, used to extract features from the target image;
[0009] Neck module, used to fuse features and construct multi-scale feature maps;
[0010] Head module, used to detect and classify feature maps and output detection results;
[0011] In the Backbone module, the Rep-Stem module is used as the input end, and is sequentially connected to the first HGBlock module, the first RepPatchEmbed module, the second HGBlock module, the second RepPatchEmbed module, three Rep-HGblock modules, the third RepPatchEmbed module, and the fourth Rep-HGblock module; the output end of the fourth Rep-HGblock module is connected to the SPPF module of the Neck module;
[0012] The Neck module includes a second Res-CSPStage module, a third Res-CSPStage module and a fourth Res-CSPStage module; the output end of the second Res-CSPStage module is connected to the CBS module in the first layer of the Head module, the output end of the third Res-CSPStage module is connected to the CBS module in the second layer of the Head module, and the output end of the fourth Res-CSPStage module is connected to the CBS module in the third layer of the Head module.
[0013] Furthermore, in the Backbone module, the output end of the second HGBlock module is simultaneously connected to the second Concat module of the Neck module; and the output end of the third Rep-HGblock module is simultaneously connected to the first Concat module of the Neck module.
[0014] Furthermore, the Neck module also includes two Upsample modules, two CBS modules, a first Res-CSPStage module and four Concat modules, wherein the SPPF module is sequentially connected to the first Upsample module, the first Concat module, the first Res-CSPStage module, the second Upsample module, the second Concat module, the second Res-CSPStage module, the first CBS module, the third Concat module, the third Res-CSPStage module, the second CBS module, the fourth Concat module and the fourth Res-CSPStage module; and the output end of the SPPF module is simultaneously connected to the fourth Concat module, and the output end of the first Res-CSPStage module is simultaneously connected to the third Concat module.
[0015] Furthermore, the structure of each layer in the Head module is the same. Each layer is first connected in sequence by a CBS module, a weight-shared convolution block, and a BN block. Then, the output end of the BN block is simultaneously connected to two ordinary convolution blocks, and finally, the output ends of the two ordinary convolution blocks are simultaneously connected to the Concat module.
[0016] Furthermore, the CBS modules are composed of a 3×3Conv block, a BN block, and a SiLU module connected in series.
[0017] Furthermore, the Rep-Stem module is composed of two Repconv modules and two SiLU modules alternately connected in series; the Repconv module is a structural reparameterization module; the Repconv module includes a 3×3Conv block, a 1×1Conv block and two BN blocks, wherein, after the 3×3Conv block is connected in parallel with the 1×1Conv block, the 3×3Conv is connected in series with a BN block, and the 1×1Conv block is connected in series with another BN block.
[0018] Furthermore, the RepPatchEmbed module includes a 7×7DWConv block, a 3×3DWConv block, two BN blocks and a CBS module, wherein the 7×7DWConv block is connected in parallel with the 3×3DWConv block, the 7×7DWConv block is connected in series with one BN block, and the 3×3DWConv block is connected in series with another BN block.
[0019] Furthermore, the working process of the Rep-Hgblock module is as follows: first, the input feature is compressed using a 1×1 CB unit channel and then sequentially sent to multiple Repconv modules to extract features; then, the output of each Repconv module is spliced together through a concat module after passing through an activation function of a SiLU module, and then the number of channels is changed through two CBS modules; finally, a shortcut block is introduced for connection to add the input information to the output information.
[0020] Furthermore, the working process of the Res-CSPStage module is as follows: first, a 1×1 CBS module is used to change the number of channels and then divided into two branches, one branch passes through two 3×3 CBS modules connected in series, and the other branch passes through a 1×1 CBS module; the outputs of the two branches pass through a concat module and then pass through a 1×1 CBS module for output.
[0021] The present invention also provides a detection method of a real-time target detection model for industrial applications, comprising the following steps:
[0022] Step S1: Send the target image to be detected into the Backbone module to complete the feature extraction of the target image, and send the features of different levels into the Neck module in three ways at the same time;
[0023] Step S2: The Neck module fuses features and enriches the expression ability of the feature map, improves the loss of detail information during feature fusion, and constructs a multi-scale feature map to output to the Head module;
[0024] Step S3: The Head module uses the features sent by the Neck module to complete the positioning and classification of the image to be detected and output the detection results.
[0025] The beneficial effects of the present invention are:
[0026] The present invention proposes a universal real-time target detection network model (YOLO-C). First, the present invention analyzes the commonly used Backbone in the YOLO network, and optimizes it from the perspective of deployment using the structural reparameterization technology. Secondly, the present invention conducts a large number of ablation experiments on commonly used Neck, Head, label allocation and loss functions to determine the network structure. Then, in order to improve the deployment effect of the network, the present invention further explores the compression performance of the model and conducts a large number of self-distillation, pruning and quantization experiments on the model. Then, in order to verify the generalization performance of the model, experiments are conducted on 7 different data sets including the COCO data set. The experimental results show that the network of the present invention is lightweight and powerful. Finally, the present invention deploys YOLO-C to the embedded devices Jeston TX2 NX and Orange Pie 5Plus, and uses the thread pool in Orange Pie 5Plus for int8 reasoning, and the reasoning speed reaches an astonishing 140+. The target detection model YOLO-C of the present invention has strong generalization performance and has good detection effects in different detection scenarios, which helps to promote the implementation of real-time target detection in industrial applications. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0028] Figure 1 Schematic diagram of the structure of the Rep-Stem module of the embodiment of the present invention.
[0029] Figure 2 Schematic diagram of the structure of the Rep-Stem module of an embodiment of the present invention; wherein (a) is the Stem used by HGNetv2, (b) is the form of Rep-Stem during training, and (c) is the form of Rep-Stem during reasoning.
[0030] Figure 3 Schematic diagram of the structure of the RepPatchEmbed module of an embodiment of the present invention; wherein (a) is the form of RepPatchEmbed during training, and (b) is the form of RepPatchEmbed during reasoning.
[0031] Figure 4 Schematic diagram of the structure of the Rep-HGblock module of the embodiment of the present invention.
[0032] Figure 5 It is a schematic diagram of the structure of the Res-CSPStage module of an embodiment of the present invention.
[0033] Figure 6 : This is a comparison chart of detection indicators of the target detection method of an embodiment of the present invention and other excellent target detection algorithms on the COCO validation set; wherein (a) is the relationship between Parameters and AP50 of the present invention on the COCO dataset, and (b) is the relationship between Flops and AP50 of the present invention on the COCO dataset. DETAILED DESCRIPTION
[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0035] The present invention discloses a real-time target detection model for industrial applications, the structure of which is as follows: Figure 1 As shown in the figure, it consists of three modules: backbone network Backbone, neck and detection head Head; among them:
[0036] The backbone network Backbone consists of a Rep-Stem module, two HGBlock modules, three RepPatchEmbed modules and four Rep-HGblock modules; among them, the Rep-Stem module is used as the input end, and is connected to the first HGBlock module, the first RepPatchEmbed module, the second HGBlock module, the second RepPatchEmbed module, the three Rep-HGblock modules, the third RepPatchEmbed module and the fourth Rep-HGblock module in sequence. The output end of the second HGBlock module is also connected to the second Concat module of the Neck module; the output end of the third Rep-HGblock module is also connected to the first Concat module of the Neck module; the output end of the fourth Rep-HGblock module is connected to the SPPF module of the Neck module. The feature maps of multiple sizes in the Backbone are fused with those in the Neck, which can improve the detection effect of the detection network on objects of different sizes.
[0037] The Neck module is flexibly composed of a SPPF module, two upsampling modules Upsample, two CBS modules, four Res-CSPStage modules and four concatenation operation Concat modules. Each CBS module is composed of a 3×3Conv and BN, SiLU in sequence. Among them, the SPPF module is connected to the first Upsample module, the first Concat module, the first Res-CSPStage module, the second Upsample module, the second Concat module, the second Res-CSPStage module, the first CBS module, the third Concat module, the third Res-CSPStage module, the second CBS module, the fourth Concat module, and the fourth Res-CSPStage module in sequence; and the output end of the SPPF module is also connected to the fourth Concat module, the output end of the first Res-CSPStage module is also connected to the third Concat module, the output end of the second Res-CSPStage module is also connected to the CBS module in the first layer of the Head module, the output end of the third Res-CSPStage module is also connected to the CBS module in the second layer of the Head module, and the output end of the fourth Res-CSPStage module is also connected to the CBS module in the third layer of the Head module; the Neck module is hierarchically connected to the Head module, which can effectively fuse feature maps from different levels, which helps to capture different scale information of the target. In object detection, small objects usually appear on higher-level feature maps, while large objects appear on lower-level feature maps. The present invention enables the model to better handle objects of different sizes through top-down and bottom-up path fusion mechanisms; at the same time, through the top-down path, higher-level abstract features can be used to enhance low-level detail features. In this way, the feature maps of each layer can be fully utilized, enhancing the expressive power of the entire network. Compared with complex multi-branch structures, the present invention achieves efficient information flow through a simple path aggregation strategy. This design not only reduces the demand for computing resources, but also improves the speed of training and reasoning.
[0038] The SPPF module generates a set of feature representations with different resolutions by applying pooling operations of different sizes to the feature map, and then cascades these representations to form a rich and multi-scale feature vector. Especially when dealing with target objects of different sizes, the SPPF module can help the model achieve better generalization ability and detection accuracy.
[0039] The Head module is divided into three layers, consisting of three CBS blocks, three weight-sharing convolution blocks nn.Con2d, three BN blocks, three Concat modules, and six ordinary convolution blocks nn.Con2d. Each layer is first connected in sequence by a CBS block, a weight-sharing convolution block, and a BN block. Then the output of the BN block is simultaneously connected to two ordinary convolution blocks, and finally the output of the two ordinary convolution blocks is simultaneously connected to the Concat module. The Head uses the shared convolution separation BN method to solve the problem of unbalanced number of learnable samples on different levels of Head, while reducing parameters and improving inference speed.
[0040] like Figure 2 As shown in (a) to (c) in the figure, the Rep-Stem module is composed of two Repconv modules and two SiLU modules connected in series. The Repconv module is a structural reparameterization module. During training, it is composed of a 3×3Conv block, a 1×1Conv block, and two BN blocks. After the 3×3Conv block is connected in parallel with the 1×1Conv block, the 3×3Conv is connected in series with a BN block, and the 1×1Conv block is connected in series with another BN block. The output ends of the two BN blocks are connected to the SiLU module together. During inference, Repcon is equivalent to 3×3Conv. This design can greatly reduce the inference delay while maintaining the detection accuracy.
[0041] Assume that the input is input and the output is output. The feature extraction process of the Rep-Stem training process is:
[0042] output=Silu(Repcon(Silu(Repcon(input)))) (1)
[0043] The reasoning process of Rep-Stem is:
[0044] output=Silu(Conv 3×3 (Silu(Conv 3×3 (input))))(2)
[0045] like Figure 3As shown in (a) to (b) in the figure, the RepPatchEmbed module is a structural reparameterization module. During training, it consists of a 7×7DWConv block, a 3×3DWConv block, two BN blocks and a CBS module. Among them, after the 7×7DWConv block is connected in parallel with the 3×3DWConv block, the 7×7DWConv block is connected in series with a BN block, and the 3×3DWConv block is connected in series with another BN block. The output ends of the two BN blocks are connected to the CBS module together. During inference, the RepPatchEmbed module is equivalent to a 7×7DWConv and a CBS. RepPatchEmbed can speed up the convergence of the model, reduce the information loss problem of the model in downsampling, and improve the learning ability of the network.
[0046] Assume that the input is F0 and the output is F1. The feature extraction process during RepPatchEmbed training is:
[0047] F1=CBS(BN(DWConv 7×7 (F0))+BN(DWConv 3×3 (F0)))(3)
[0048] The reasoning process of RepPatchEmbed is:
[0049] F1=CBS(DWConv 7×7 (F0))(4)
[0050] like Figure 4 As shown in Figure 1, Rep-HGblock consists of three CB modules, two CBS modules, n (2≤n≤4) Repcon, four ADD, three SiLU and one concat. n is related to the model size.
[0051] Rep-HGblock first compresses the input features using a 1×1 CB unit and then sequentially sends them to n (2≤n≤4) Repconvs to extract features. The output of each Repconv is concatted together after passing through a SiLU activation function, and then the number of channels is changed through two CBS modules. Finally, a shortcut connection is introduced to add the input information to the output information.
[0052] This design runs multiple gradient flow branches in parallel, obtaining richer gradient information while maintaining lightweight, thereby improving convergence speed. At the same time, the network can contain combinations of receptive fields and center steps of different sizes, thereby satisfying the detection of objects of different scales.
[0053] like Figure 5As shown, Res-CSPStage consists of three 1×1 convolutional CBSs, two 3×3 convolutional CBSs and one concat.
[0054] Res-CSPStage first uses a 1×1 CBS to change the number of channels and then divides it into two branches. One branch passes through two 3×3 CBSs in series, and the other branch passes through a 1×1 CBS. The outputs of these two branches are concat and then output through a 1×1 CBS. This design uses cross-level dense connections to enhance the network's feature learning ability, reduce memory access costs, and avoid the large number of parameters and computational burdens brought by a large number of 3×3 convolutional layers.
[0055] The present invention also provides a detection method for real-time target detection for industrial applications, which uses the COCO dataset training set for training and the COCO dataset verification set for verification. The real-time target detection model is trained with the training set, and the trained model is used to detect the image to be detected. The specific steps are as follows:
[0056] S1, send the image to be detected to the backbone network Backbone, complete the feature extraction of the image, and send the shallow and deep features to the neck in three ways at the same time;
[0057] Backbone sends the image to be detected to two sequentially connected groups of HGBlock+RepPatchEmbed blocks to achieve 16 times downsampling after Rep-Stem completes 4 times downsampling. Then it is sent to three series-connected Rep-HGblocks to further extract features. Finally, the above output is sent to the RepPatchEmbed+Rep-HGblock block to achieve 32 times downsampling and improve the detection effect of large targets. The design of Backbone makes the model lightweight and has extremely fast inference speed, which is easy to deploy in embedded devices. At the same time, it contains a combination of receptive fields of different sizes and center steps to meet the detection of objects of different scales.
[0058] S2, Neck, consists of an SPPF, multiple Upsample, Concat, Res-CSPstage, and CBS. Upsample is responsible for upsampling feature maps so that deep features can be concat with shallow features, and Res-CSPstage is responsible for fusing features of different scales. CBS is responsible for downsampling feature maps to achieve fusion of shallow information with deep information. The design of Neck can fuse different features, enrich the expression ability of feature maps, improve the loss of detail information during feature fusion, and construct multi-scale feature maps to output to Head, so as to achieve detection of objects of various sizes through Head;
[0059] S3, head, mainly consists of three CBS, three shared weight separation BN convolutions and six convolutions. CBS is responsible for unifying feature maps with different channel numbers into the same number of channels. The three shared weight separation BN convolutions are designed to solve the problem of unbalanced number of learnable samples on different levels of heads. At the same time, the use of shared heads can reduce parameters and improve inference speed. The last six convolutions constitute three detection heads to detect objects of different sizes. Head uses the features sent by Neck to complete the positioning and classification of the image to be detected to output the detection results.
[0060] In order to verify the detection effect of the target detection model of the present invention (i.e., YOLO-C below), a comparative experiment was carried out. The dataset used the public dataset NEU-DET of Northeastern University and the COCO 2017 dataset, in which the NEU-DET dataset was enhanced, and the enhancement methods included scaling, distortion, rotation, and color gamut transformation. The enhanced dataset has a total of 5372 images, and the dataset is randomly divided into training set, validation set, and test set in a ratio of 7:1:2.
[0061] On the NEU-DET dataset, the six classic detection networks compared are Yolov3 (see Joseph Redmonand Ali Farhadi.YOLOv3:An incremental improvement.arXiv preprint arXiv:1804.02767,2018.), Yolov 5 (see G.Jocher.YOLOv5-Master.Accessed:Mar.1,2021.[Online].https: / / github.com / ultralytics / yolov5), Yolov7 (see Wang CY,Bochkovskiy A,Liao HY M.YOLOv7:Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[J].arXiv preprint arXiv:2207.02696,2022), TPH-YOLOv5 (see Zhu X,Lyu S,Wang X,et al.TPH-YOLOv5:Improved YOLOv5based on transformer prediction head for object detection on drone-capturedscenarios[C] / / Proceedings of the IEEE / CVF international conference oncomputer vision.2021:2778-2788.), RT-DETR(Wenyu Lv,Shangliang Xu,Yian Zhao,Guanzhong Wang,Jinman Wei,Cheng Cui,Yuning Du,Qingqing Dang,and Yi Liu.DETRsbeat YOLOs on real-time object detection.arXiv preprint arXiv:2304.08069,2023.1,3,7,8,2,4), Yolov8(see Jocher Glenn.Ultralytics yolov8.https: / / github.com / ultralytics / ultralytics,2023). The experimental results are shown in Table 1, where bold represents the best indicators.
[0062] Table 1 Comparison experimental results of different detection models
[0063]
[0064] This embodiment refers to Yolov62.0, and controls the models such as n, t, s, and m through width factors and depth factors. Table 1 is the experimental results of the enhanced NEU-DET data set. It can be seen from the experimental results that compared with RT-DETR, the parameter amount of the present invention is 4.8% of it, the calculation amount is 9.3% of it, and the AP is still 0.5% higher than it. Compared with Yolov5n of the same order of magnitude, the map of this embodiment is 10.1% higher than it. Compared with Yolov8n, the parameter amount of the n model of this embodiment is 65% of its n model, and the calculation amount is 64% of it. The map of this embodiment is still 3% higher than it, and the reasoning time is 5% better than it, which shows the powerful performance of the network of this embodiment in defect detection on the steel surface. The detector of the DETR series is developed based on Transformer. The model of Transformer is more complex than Cnn and relies on a large amount of training data. The detection head of Yolov8 occupies a large amount of parameters and calculations in its model, while the detection head with shared parameters in YOLO-C of the present invention reduces a large amount of parameters and calculations. At the same time, the detection heads that share parameters can solve the problem of unbalanced number of learnable samples on different levels of heads on small and medium data sets. In the detection of defects such as "Cr" that are extremely similar to the background, the network of this embodiment shows more powerful performance, far superior to other networks. The reason is that when faced with a large amount of irrelevant noise and complex background, other networks have insufficient feature extraction capabilities and small receptive fields. The features extracted by the neural network are reduced, and the network may fit noise rather than features. The introduction of Rep-Hgblock can make the network contain a combination of receptive fields of different sizes and center step sizes, thereby improving detection accuracy. From the above experimental results, it can be seen that the YOLO-Cn network achieves a better balance between detection accuracy and model size.
[0065] Table 2 shows the performance of YOLO-C on the COCO dataset and compares it with six classic detection networks. It can be seen that YOLO-C's Params and FLOPs are better than other networks, reflecting its good performance on large datasets. Although the scale of YOLO-Cn is similar to that of YOLOv5-N, its AP50 is 6.5% higher than that of YOLOv5-N. The YOLO-Cs model is similar in scale to PPYOLOE-S, but its AP50 is 1.3% higher than that of PPYOLOE-S. These illustrate the rationality of the network design of this embodiment.
[0066] Table 2 Experimental results of YOLO-C and other excellent detection networks on COCO2017 dataset
[0067] Methods <![CDATA[AP val50 ]]> FLOPs(G) Params(M) YOLOv5-N 45.7% 4.5G 1.9M YOLOv5-S 56.8% 16.5G 7.2M YOLOv5-M 64.1% 49.0 G 21.2M YOLOX-Tiny 50.3% 6.5G 5.1M YOLOX-S 59.3% 26.8G 9.0M YOLOX-M 65.6% 73.8 G 25.3M PPYOLOE-S 59.6% 17.4G 7.9M PPYOLOE-M 65.9% 49.9 G 23.4M YOLOv7-Tiny 55.2% 13.7G 6.2M YOLOv7 69.7% 104.7 G 36.9M YOLOv6-3.0-N 52.7% 11.4G 4.7M YOLOv6-3.0-S 61.2% 45.3 G 18.5M YOLOv6-3.0-M 66.1% 85.8 G 34.9M YOLOv8-N 52.6% 8.7G 3.2M YOLOv8-S 61.8% 28.6 G 11.2M YOLOv8-M 67.2% 78.9 G 25.9M YOLO-CN 52.2% 5.4 G 2.01M YOLO-Cs 60.9% 18.5 G 7.16M YOLO-Cm 64.7% 39.2 G 12.2M
[0068] like Figure 6 (a) to (b) and Table 2 show the detection results of the YOLO-C target detection model of the present invention and other advanced target detectors on the MS-COCO 2017 dataset, including: YOLOv5 (see G. Jocher. YOLOv5-Master. Accessed: Mar. 1, 2021. [Online]. https: / / github.com / ultralytics / yolov5), YOLOX (see Ge Z, Liu S, Wang F, et al. Yolox: Exceeding yolo series in 2021 [J]. arxiv preprint arxiv: 2107.08430, 2021.), PPYOLOE (see Xu S, Wang X, Lv W, et al. PP-YOLOE: An evolved version of YOLO [J]. arxiv preprint arxiv: 2203.16250, 2022.), and Yolov7 (see Wang CY, Bochkovskiy A, Liao HY M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors [J]. arXiv preprint arXiv: 2207.02696, 2022), YOLOv6-3.0 (see Li C, Li L, Geng Y, et al. Yolov6v3.0: A full-scale reloading [J]. arxiv preprint arxiv: 2301.05586, 2023), Yolov8 (see Jocher Glenn. Ultralytics yolov8. https: / / github.com / ultralytics / ultralytics, 2023). It can be seen from the test results that the model of the present invention outperforms other advanced networks in Parameters-AP50 and FLOPs-AP50 on the COCO dataset. At the same time, the network deployment of this embodiment is friendly.
[0069] The network of the present invention has smaller parameters and lower model complexity, and the network of the present invention has better detection effects than other excellent networks on 7 different data sets, especially small industrial defect detection data sets. It has excellent performance on both large and small data sets, which also shows that the network of the present invention can not only learn useful features from large amounts of data, but also extract key information even in small-scale data sets. At the same time, it shows that the model of the present invention is highly adaptable, can be easily expanded to other fields without too much adjustment, is insensitive to changes in data scale, can work stably in different scenarios, and will not cause significant fluctuations in performance due to changes in the amount of training data. All of these show the rationality of the network design of the present invention, and it is suitable for application in industrial detection.
[0070] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0071] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims
1. A real-time target detection model for industrial applications, characterized in that: include: Backbone module, used to extract features from the target image; Neck module, used to fuse features and construct multi-scale feature maps; Head module, used to detect and classify feature maps and output detection results; In the Backbone module, the Rep-Stem module is used as the input end, and is sequentially connected to the first HGBlock module, the first RepPatchEmbed module, the second HGBlock module, the second RepPatchEmbed module, three Rep-HGblock modules, the third RepPatchEmbed module, and the fourth Rep-HGblock module; the output end of the fourth Rep-HGblock module is connected to the SPPF module of the Neck module; The Neck module includes a second Res-CSPStage module, a third Res-CSPStage module and a fourth Res-CSPStage module; the output end of the second Res-CSPStage module is connected to the CBS module in the first layer of the Head module, the output end of the third Res-CSPStage module is connected to the CBS module in the second layer of the Head module, and the output end of the fourth Res-CSPStage module is connected to the CBS module in the third layer of the Head module.
2. A real-time target detection model for industrial applications according to claim 1, characterized in that: In the Backbone module, the output end of the second HGBlock module is simultaneously connected to the second Concat module of the Neck module; and the output end of the third Rep-HGblock module is simultaneously connected to the first Concat module of the Neck module.
3. A real-time target detection model for industrial applications according to claim 1, characterized in that: The Neck module also includes two Upsample modules, two CBS modules, a first Res-CSPStage module and four Concat modules, wherein the SPPF module is sequentially connected to the first Upsample module, the first Concat module, the first Res-CSPStage module, the second Upsample module, the second Concat module, the second Res-CSPStage module, the first CBS module, the third Concat module, the third Res-CSPStage module, the second CBS module, the fourth Concat module and the fourth Res-CSPStage module; and the output end of the SPPF module is simultaneously connected to the fourth Concat module, and the output end of the first Res-CSPStage module is simultaneously connected to the third Concat module.
4. A real-time target detection model for industrial applications according to claim 1, characterized in that: The structure of each layer in the Head module is the same. Each layer is first connected in sequence by a CBS module, a weight-sharing convolution block, and a BN block. Then the output end of the BN block is simultaneously connected to two ordinary convolution blocks. Finally, the output ends of the two ordinary convolution blocks are simultaneously connected to the Concat module.
5. A real-time target detection model for industrial applications according to any one of claims 1 to 4, characterized in that: The CBS modules are composed of a 3×3 Conv block, a BN block, and a SiLU module connected in series.
6. A real-time target detection model for industrial applications according to claim 1, characterized in that: The Rep-Stem module is composed of two Repconv modules and two SiLU modules alternately connected in series; the Repconv module is a structural reparameterization module; the Repconv module includes a 3×3Conv block, a 1×1Conv block and two BN blocks, wherein the 3×3Conv block is connected in parallel with the 1×1Conv block, the 3×3Conv is connected in series with a BN block, and the 1×1Conv block is connected in series with another BN block.
7. A real-time target detection model for industrial applications according to claim 1, characterized in that: The RepPatchEmbed module includes a 7×7DWConv block, a 3×3DWConv block, two BN blocks and a CBS module, wherein the 7×7DWConv block is connected in parallel with the 3×3DWConv block, the 7×7DWConv block is connected in series with one BN block, and the 3×3DWConv block is connected in series with another BN block.
8. A real-time target detection model for industrial applications according to claim 1 or 6, characterized in that: The working process of the Rep-Hgblock module is as follows: first, the input features are compressed using a 1×1 CB unit channel and then sequentially sent to multiple Repconv modules to extract features; The output of each Repconv module is then concatenated together through the concat module after passing through the activation function of the SiLU module, and then passed through two CBS modules to change the number of channels; finally, the shortcut block is introduced for connection to add the input information to the output information.
9. A real-time target detection model for industrial applications according to any one of claims 1 to 3, characterized in that: The working process of the Res-CSPStage module is as follows: first, a 1×1 CBS module is used to change the number of channels and then divided into two branches, one branch passes through two 3×3 CBS modules connected in series, and the other branch passes through a 1×1 CBS module; the outputs of the two branches pass through a concat module and then pass through a 1×1 CBS module for output.
10. A detection method for a real-time target detection model for industrial applications as claimed in any one of claims 1 to 7, characterized in that: The following steps are involved: Step S1: Send the target image to be detected into the Backbone module to complete the feature extraction of the target image, and send the features of different levels into the Neck module in three ways at the same time; Step S2: The Neck module fuses features and enriches the expression capability of feature maps, improves the loss of detail information during feature fusion, and constructs multi-scale feature maps to output to the Head module. Step S3: The Head module uses the features sent by the Neck module to complete the positioning and classification of the image to be detected and output the detection result.
