Detection model training method, detection method, equipment, storage medium and product

Abstract

The present invention discloses a detection model training method, a detection method, a device, a storage medium and a product, and relates to the field of target detection. The training method comprises obtaining a sample data set; constructing a target detection model, wherein the target detection model is an improved YOLOv6 model, wherein the improved YOLOv6 model is obtained by replacing a reparameterization module with a fast-reparameterization module on the basis of the original YOLOv6 model; the fast-reparameterization module comprises a partially reparameterized convolution block, two convolution modules and an additive fusion module connected in sequence, wherein the additive fusion module is connected to an input end of the partially reparameterized convolution block through a jump connection; and the target detection model is trained using a sample data set to obtain a trained target detection model. The present invention can reduce the model volume and the amount of parameters while ensuring that the target detection accuracy does not decrease, thereby improving the target detection speed.

Description

Technical Field

[0001] The present invention belongs to the field of target detection technology, and in particular relates to a detection model training method, a detection method, a device, a storage medium and a product. Background Art

[0002] With the rapid development of IoT technology, the YOLO (You Only Look Once) algorithm has become an important tool in the field of computer vision due to its high accuracy, fast detection, and flexible structure. After multiple version iterations, the YOLO algorithm has evolved to YOLOv10. However, as the detection accuracy continues to improve, the scale of the model has also expanded, which has brought great challenges to the deployment of deep learning models on edge devices. Edge devices usually have low computing power, memory capacity, and limited energy consumption, making it difficult for deep learning models to run efficiently.

[0003] In order to address the challenges faced by edge devices (or resource-constrained terminal devices) when deploying deep learning models, researchers have proposed a variety of lightweight improvement schemes aimed at reducing the consumption of computing resources while maintaining a certain detection accuracy, so that the YOLO algorithm can be effectively applied to embedded edge devices and provide high real-time and security. These lightweight methods not only improve the real-time performance of the model, but also enhance its security and applicability in fields such as the Internet of Things, intelligent monitoring, and unmanned driving. In recent years, lightweight neural networks such as MobileNet, ShuffleNet, GhostNet, and FasterNet have been proposed one after another, providing more diverse technical paths for model optimization. At present, the mainstream lightweight improvement schemes mainly include the following:

[0004] (1) Use lightweight networks to replace the feature extraction module of YOLO: This method significantly reduces the computational complexity and parameters of the model and improves the inference speed by replacing the original feature extraction module in the YOLO model with a lightweight network (such as MobileNet, ShuffleNet, etc.). Although this method can effectively reduce the computational requirements of the model and improve the inference speed, the feature extraction capabilities of these lightweight networks are relatively weak, especially when dealing with complex scenes or high-resolution inputs, the detection accuracy will drop significantly, making it difficult for this method to balance speed and accuracy in all application scenarios. That is, the technical trade-off between lightweight and high accuracy always exists.

[0005] (2) Pruning and knowledge distillation techniques: Pruning techniques reduce the size of models by removing redundant parameters, thereby speeding up inference; while knowledge distillation techniques optimize the performance of small models by transferring knowledge from large models to small models. These methods can improve inference speed while maintaining a certain level of accuracy. However, the implementation process of these two techniques is relatively complex. The effects of pruning and knowledge distillation rely on complex parameter debugging, and the results are uncertain. In actual deployment, especially on edge devices, the stability and actual effects of these methods are difficult to fully predict.

[0006] Although the lightweight YOLO model can adapt to edge devices by reducing the number of parameters and model size, the hardware performance of edge devices still has bottlenecks and cannot fully support the efficient operation of complex deep learning models. This is because deep learning models inherently require a large number of floating-point operations. Even by optimizing the network structure and reducing redundant calculations, the problem of the inference speed bottleneck cannot be fundamentally solved, that is, the lightweight design can only improve the inference speed to a limited extent. In addition, the hardware resources of edge devices may still become a limiting factor during the inference process, especially when the input data scale is large or the task is more complex. Even if the model is optimized, it is difficult to achieve ideal real-time performance.

[0007] Most lightweight neural network designs focus on "lightness", that is, lightweighting is achieved by reducing the model size and the number of floating-point operations. However, this lightweight design does not necessarily lead to a similar reduction in latency. This is mainly because the floating-point operations per second of many lightweight modules are inefficient, and the main reason for the low efficiency is the frequent memory access of the operation. Most of these methods only achieve lightweighting in terms of model size, and the inference speed has not been significantly improved, which cannot fully meet the application scenarios with high real-time requirements.

[0008] Existing lightweight accelerated reasoning solutions, such as replacing YOLO's original backbone feature extraction network with a lightweight backbone network, can reduce the model size and improve the reasoning speed to a certain extent, but this method also brings significant accuracy loss. The feature extraction capability of the lightweight backbone network is weaker than that of YOLO's original backbone network, resulting in a significant decrease in detection accuracy, resulting in a situation where speed is increased at the expense of accuracy. Therefore, how to reduce the model size and improve the reasoning speed while ensuring the detection accuracy of the model is still a technical problem that has not yet been fully solved. Summary of the invention

[0009] The purpose of the present invention is to provide a detection model training method, detection method, device, storage medium and product to solve the problem that traditional lightweight methods are difficult to reduce the model size, improve the inference speed and ensure the detection accuracy of the model.

[0010] The present invention solves the above technical problems through the following technical solutions: a target detection model training method, comprising:

[0011] Obtain a sample data set; wherein the sample data set includes a plurality of labeled training images, and the labels are real frames of targets in the training images;

[0012] Constructing a target detection model; wherein the target detection model is an improved YOLOv6 model, the improved YOLOv6 model includes a backbone network, a neck network and a detection network, and on the basis of the original YOLOv6 model, a fast-reparameterization module is used to replace the reparameterization modules in the backbone network and the neck network; the fast-reparameterization module includes a partially reparameterized convolution block, two convolution modules and an additive fusion module connected in sequence, and the additive fusion module is also connected to the input end of the partially reparameterized convolution block through a jump connection;

[0013] Using the sample data set to train the target detection model to obtain a trained target detection model;

[0014] The target detection model is trained using the sample data set, including:

[0015] Inputting the training image into the target detection model, and using the backbone network to extract features of the training image to obtain feature maps P2, P3, P4, and P5 of different scales;

[0016] The neck network is used to fuse low-level semantic features and high-level semantic features in feature maps P2, P3, P4, and P5 to obtain fused feature maps P3', P4', and P5' of different scales;

[0017] Using the detection network to perform target detection on the fused feature map P3', the fused feature map P4' and the fused feature map P5', respectively, to obtain prediction frames of different sizes;

[0018] A loss value between the predicted box and the corresponding real box is calculated, and the parameters of the target detection model are adjusted according to the loss value.

[0019] By replacing the reparameterization module in the original YOLOv6 model with a fast-reparameterization module, the target detection model of the present invention significantly improves the computational efficiency. The fast-reparameterization module is based on the design of the partial reparameterization convolution block, which not only maintains the lightweight of the calculation, but also realizes the information interaction between channels through two convolution modules, and more efficiently utilizes all channel feature information; the partial reparameterization convolution block significantly reduces the amount of calculation and the number of memory accesses while maintaining the feature expression capability.

[0020] Furthermore, the partially re-parameterized convolution block is used to perform a convolution operation on the 1 / 2 channel feature, and output the convolution operation result after splicing with the remaining 1 / 2 channel feature;

[0021] Each of the convolution modules includes a convolution layer, a batch normalization layer and an activation function connected in sequence.

[0022] Furthermore, the partially reparameterized convolution block includes 1 RepVGG Block, 1 Identity module and 1 splicing module, the RepVGG Block is used to perform a reparameterized convolution operation on the 1 / 2 channel features, the Identity module is used to directly output the remaining 1 / 2 channel features, and the outputs of the RepVGG Block and the Identity module are merged into the final output through the splicing operation of the splicing module.

[0023] Furthermore, the backbone network adopts an EfficientRep structure, and the neck network adopts a RepBi-PAN structure.

[0024] Further, the backbone network includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module, a fifth feature extraction module and a SimCSPSPPF module connected in sequence; wherein the first feature extraction module includes 1 RepVGG Block, the second feature extraction module includes 1 RepVGG Block and 1 fast-reparameterization module, the third feature extraction module includes 1 RepVGG Block and 2 fast-reparameterization modules, the fourth feature extraction module includes 1 RepVGG Block and 2 fast-reparameterization modules, and the fifth feature extraction module includes 1 RepVGG Block and 1 fast-reparameterization module;

[0025] The first feature extraction module is used to extract features from the training image to obtain a feature map P1; the second feature extraction module is used to extract features from the feature map P1 to obtain a feature map P2; the third feature extraction module is used to extract features from the feature map P2 to obtain a feature map P3; the fourth feature extraction module is used to extract features from the feature map P3 to obtain a feature map P4; the fifth feature extraction module is used to extract features from the feature map P4 to obtain a feature map P5;

[0026] In the neck network, two fast-reparameterization modules connected in sequence are used to replace the original reparameterization module.

[0027] In the P2 to P5 layers of the backbone network, 1, 2, 2, and 1 fast-reparameterization modules are stacked respectively to achieve a balance between accuracy and speed; in the P3' to P5' layers of the neck network, 2 fast-reparameterization modules are stacked in each layer to replace the original reparameterization modules; through the application of fast-reparameterization modules in the backbone network and the neck network, the target detection model greatly improves the inference speed and reduces the accuracy loss while maintaining good feature extraction and fusion capabilities.

[0028] Furthermore, in the training stage, each of the RepVGG Blocks includes a first branch, a second branch, a third branch and a ReLU activation function, and the first branch, the second branch and the third branch are connected to the ReLU activation function through an adder; the first branch and the second branch both include a convolution layer and a batch normalization layer, and the third branch is a batch normalization layer;

[0029] After training, the reparameterization technique is used to simplify the first branch, second branch, and third branch of each RepVGG Block into a reparameterized convolutional block for efficient inference.

[0030] Furthermore, the SimCSPSPPF module includes a first convolution module, a second convolution module, a third convolution module, a first maximum pooling layer, a second maximum pooling layer, a third maximum pooling layer, a first splicing layer, a fourth convolution module, a fifth convolution module, a second splicing layer and a sixth convolution module connected in sequence; the third convolution module, the first maximum pooling layer, the second maximum pooling layer are also connected to the first splicing layer, and the input end of the first convolution module is also connected to the second splicing layer through the seventh convolution module.

[0031] Furthermore, an improved border regression loss function is used to calculate the loss value between the predicted box and the corresponding real box; the improved border regression loss function is:

[0032] ;

[0033] ;

[0034] ;

[0035] ;

[0036] ;

[0037] ;

[0038] ;

[0039] Among them, L represents the loss value, IoU represents the intersection-over-union ratio between the real box and the predicted box, represents the distance loss, Ω represents the shape loss, β 1 、β 2 represent the learnable weight coefficients, ρ represents the Euclidean distance between the center point of the real box and the predicted box, c represents the diagonal length of the minimum circumscribed rectangle of the predicted box and the real box, α represents the positive balance parameter, Indicates the consistency of the aspect ratio between the predicted box and the real box, , , , , Both represent intermediate quantities. , Represents the coordinates of the center point of the real box, , Represents the center point coordinates of the prediction box, , Represents the width and height of the minimum bounding rectangle of the predicted box and the real box respectively, θ represents the degree of attention to shape loss, , Respectively represent the width of the predicted box and the real box, , Represent the height of the predicted box and the true box respectively.

[0040] Based on the same concept, the present invention provides a target detection method, comprising:

[0041] Acquire the image to be detected;

[0042] Obtain a trained target detection model using the target detection model training method described above;

[0043] The trained target detection model is used to perform target detection on the image to be detected.

[0044] Based on the same concept, the present invention also provides an electronic device, including a memory, a processor, and a computer program / instructions stored in the memory, wherein the processor executes the computer program / instructions to implement the target detection model training method or target detection method as described above.

[0045] Based on the same concept, the present invention also provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the target detection model training method or target detection method as described above.

[0046] Based on the same concept, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the target detection model training method or target detection method as described above.

[0047] Beneficial Effects

[0048] Compared with the prior art, the advantages of the present invention are:

[0049] Based on the original YOLOv6 model, the present invention adopts a fast-reparameterization module to replace the reparameterization module to obtain a target detection model. The partial reparameterization convolution blocks in the fast-reparameterization module can effectively improve the reasoning speed by reducing the convolution calculation amount while still effectively maintaining the ability of feature extraction and expression; the two convolution modules in the fast-reparameterization module realize information interaction between different channels, more effectively utilize the information of all channels, improve the feature extraction ability, and thus improve the target detection accuracy. The target detection model of the present invention realizes the lightweight of model volume and parameters without reducing the target detection accuracy, realizes a significant improvement in reasoning speed, and improves the target detection speed; compared with the existing lightweight solution, the present invention avoids the trade-off problem between speed and accuracy through a more effective model structure design, so that the target detection model can realize high-speed reasoning on the edge device and maintain high-precision detection, and improves the deployment efficiency of the model on the edge device with limited computing power, memory and energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0051] Figure 1 is a flow chart of the target detection model training method in an embodiment of the present invention;

[0052] Figure 2It is the original YOLOv6 model architecture diagram in the embodiment of the present invention;

[0053] Figure 3 It is a diagram of the improved YOLOv6 model architecture in an embodiment of the present invention;

[0054] Figure 4 1 is a diagram of the architecture of the Faster-RepBlock module in an embodiment of the present invention;

[0055] Figure 5 : is the architecture diagram of the convolution module ConvModule_N in the embodiment of the present invention; wherein h represents the height of the feature map, w represents the width of the feature map, c p Indicates the number of channels;

[0056] Figure 6 1 is a diagram of the architecture of a partially re-parameterized convolution block PRepConv in an embodiment of the present invention;

[0057] Figure 7 is a diagram of the architecture of the RepVGG Block in the training phase in an embodiment of the present invention;

[0058] Figure 8 is a diagram of the RepVGG Block architecture in the reasoning phase of an embodiment of the present invention;

[0059] Fig. 9 It is a diagram of the SimCSPSPPF module architecture in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The following is a clear and complete description of the technical solutions in the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0062] Example 1

[0063] Figure 1 The flowchart of the target detection model training method proposed in the present invention is shown. Figure 1 As shown, the target detection model training method includes the following steps:

[0064] Step 1: Obtain a sample data set; the sample data set contains multiple labeled training images; the labels are the true boxes of the targets in the training images;

[0065] Step 2: Build a target detection model; the target detection model is an improved YOLOv6 model, which includes a backbone network Backbone, a neck network Neck, and a detection network Head. Based on the original YOLOv6 model, a fast-reparameterization module is used to replace the reparameterization modules in the backbone network Backbone and the neck network Neck.

[0066] Step 3: Use the sample data set in step 1 to train the target detection model in step 2 to obtain a trained target detection model.

[0067] The YOLOv6 model not only maintains real-time detection capabilities, but also improves detection accuracy, which is very important for industrial applications that require efficient and accurate detection. In addition, the network architecture and optimization technology used by the YOLOv6 model make it more efficient in embedded devices and edge computing environments, which helps to deploy in resource-constrained industrial environments. Therefore, the target detection model of the present invention is obtained by improving the original YOLOv6 model.

[0068] Figure 2 The architecture diagram of the original YOLOv6 model is shown. First, the original YOLOv6 model designs an efficient re-parameterized backbone network Backbone based on RepVGG. The backbone network Backbone is mainly composed of re-parameterization modules (Re-parameterization Block, RepBlock), and each RepBlock is composed of multiple RepVGG Blocks in series. In the training phase, the RepVGG Block performs convolution operations through three parallel branches, which are BN (Batch Normalization, normalized batch layer), Conv1×1 (i.e. 1×1 convolution) + BN and Conv3×3 (i.e. 3×3 convolution) + BN; in the inference phase, the RepVGG Block in each RepBlock is simplified to a stack of re-parameterized convolution blocks RepConv (Conv3×3+ReLU), in which the 3×3 convolution is highly optimized on mainstream GPUs (Graphics Processing Units, image processors) and CPUs (Central Processing Units / Processors, central processing units), and has higher computational density. Therefore, the backbone network fully utilizes the computing power of the hardware, significantly reducing the inference latency while enhancing the representation capability of the model.

[0069] Secondly, in order to further reduce hardware latency, the original YOLOv6 model introduced the Rep structure in the feature fusion structure of the neck network Neck. Drawing on the work of FPN, PANet, BiFPN and PRB-FPN, the original YOLOv6 model designed an improved PAN structure - RepBi-PAN. The RepBi-PAN structure integrates the feature maps of three adjacent layers through the bidirectional concatenation (BiC) module, fuses a low-level feature from the main trunk into the current feature layer, and uses the SimCSPSPPF module (Simplified Convolutional Neural Network with Concatenated Spatial Pyramid Pooling, simplified cross-stage partial spatial pyramid pooling-fast module) to enhance the feature representation capability. In addition, the cross-stage partial connection module CSPBlock is replaced by RepBlock, and the width and depth of the model are adjusted accordingly.

[0070] Finally, the original YOLOv6 model decouples the detection head to optimize bounding box regression and category classification respectively. Based on the decoupled detection head of FCOS (i.e., fully convolutional one-stage object detector) and YOLOX, the original YOLOv6 model introduces a hybrid channel strategy and reduces the number of 3×3 convolutional layers to one, reducing the computational complexity of the head. The width of the head is determined by the width multiplier of the backbone network and the neck. These improvements significantly reduce the computational cost, thereby further shortening the inference delay.

[0071] The present invention improves the original YOLOv6 model by replacing the RepBlock in the original YOLOv6 model with a Faster-Re-parameterization Block (Faster-RepBlock) to obtain an improved YOLOv6 model. Figure 3 As shown, Faster RepBlock*n represents a series structure of n FasterRepBlocks. The present invention uses the improved YOLOv6 model as the target detection model, and adopts the lightweight and efficient fast-reparameterization module Faster-RepBlock to replace the RepBlock in the original YOLOv6 model, realizing a lightweight backbone network Backbone and neck network Neck, and effectively optimizing network efficiency and accelerating model reasoning by reducing computational redundancy and memory access times.

[0072] like Figure 4As shown in FIG, Faster-RepBlock includes a partially re-parameterized convolution block (Partial Re-parameterization Convolution, PRepConv), two convolution modules ConvModule and an additive fusion module connected in sequence. The additive fusion module is also connected to the input end of the partially re-parameterized convolution block PRepConv through a jump connection. Figure 5 As shown, each convolution module ConvModule includes a convolution layer Conv, a batch normalization layer BN and a ReLU activation function connected in sequence.

[0073] The backbone network FasterNet discloses a lightweight convolution block Partial Convolution (PConv), which extracts spatial features more efficiently by reducing redundant calculations and memory accesses at the same time. PConv applies ordinary convolution to only a portion of the input channels (for example, 1 / 4 channels) to extract spatial features, while the remaining channels remain unchanged. To achieve continuous memory access, cp consecutive channels are selected as representatives to calculate the entire feature map. The core of PConv is to optimize resource utilization by avoiding repeated calculations of all channels. Despite the reduced amount of calculations, the backbone network FasterNet can still effectively capture spatial features and maintain the accuracy and performance of the model. It is especially suitable for resource-constrained scenarios such as edge computing and embedded devices, while improving the inference speed.

[0074] Due to the significant similarity between different channels, the feature maps in target detection usually have a certain degree of redundancy. Therefore, the efficiency of PConv convolution is re-analyzed, and the present invention proposes a more efficient partial reparameterized convolution block PRepConv as the core operator. The partial reparameterized convolution block PRepConv performs convolution operation on the 1 / 2 channel features of the input feature map, and splices the convolution operation result with the remaining 1 / 2 channel features and outputs them. Figure 6 As shown in the figure, the partial reparameterized convolution block contains 1 RepVGG Block, 1 Identity module and 1 splicing module. The RepVGG Block is used to perform reparameterized convolution operations on the 1 / 2 channel features, and the Identity module is used to directly output the remaining 1 / 2 channel features. The outputs of the RepVGG Block and the Identity module are combined into the final output through the splicing operation of the splicing module. The Identity module is a simple activation function (i.e., f(x)=x) that does not perform any transformation on the input. That is, in the Identity module, the remaining, unconvolved 1 / 2 channel features are output without any operation.

[0075] In the training phase, the RepVGG Blcok in the partially reparameterized convolution block PRepConv uses three branches, BN, Conv1×1+BN, and Conv3×3+BN, to extract feature maps of some input channels respectively; in the inference phase, the three branches are simplified to RepConv+ReLU through the structural reparameterization method, achieving efficient inference performance. The partially reparameterized convolution block PRepConv uses the reparameterized convolution of RepVGGBlcok to extract features from some channel feature maps, which significantly reduces the amount of calculation and memory access times while maintaining the feature expression ability, and improves the inference speed.

[0076] The partially re-parameterized convolution block PRepConv cannot realize the information interaction between different channels of the feature map, and therefore fails to make full use of the information of all channels. To solve this problem, the present invention adds two convolution modules ConvModule_1 and an addition fusion module on the basis of PRepConv. PRepConv performs convolution operation on the 1 / 2 channel features of the input feature map, and splices the convolution operation result with the remaining 1 / 2 channel features and outputs it to the first convolution module ConvModule_1; the first convolution module ConvModule_1 doubles the number of channels through 1×1 convolution Conv, and then after batch normalization BN and ReLU activation processing, it is input to the second ConvModule_1; the second ConvModule_1 reduces the number of channels back to the original dimension through 1×1 convolution Conv, and finally after batch normalization BN and ReLU activation processing, it is added to the Faster-RepBlock input residual through an addition fusion module. Faster-RepBlock optimizes the feature maps of continuous channels and selects the first or last continuous channel as the representative of the entire feature map for calculation, further reducing the floating point operations (FLOPs) required for convolution operations. Faster-RepBlock not only maintains lightweight calculations and better utilizes the computing resources of the device, but also performs well in extracting spatial features. It also realizes information interaction between different channels through two convolution modules ConvModule, thereby more effectively utilizing the information of all channels and improving detection accuracy.

[0077] The backbone network Backbone extracts features from the training images to obtain feature maps P2, P3, P4 and P5 of different scales, and passes the feature maps P2, P3, P4 and P5 to the neck network Neck for feature fusion. The neck network Neck fuses the low-level semantic features and high-level semantic features in the feature maps P2, P3, P4 and P5 to generate fused feature maps P3', P4' and P5' of three scales, and passes the fused feature maps P3', P4' and P5' to the detection network Head for detection. The detection network Head contains three detection modules Detect of different scales, which perform target detection on the fused feature maps P3', P4' and P5' transmitted from the neck network Neck respectively.

[0078] In a specific embodiment of the present invention, the backbone network Backbone adopts the EfficientRep structure, and the neck network Neck adopts the RepBi-PAN structure. Both the EfficientRep structure and the RepBi-PAN structure are based on the RepBlock design. This structure uses a multi-branch topology during training, but can be fused into a single 3×3 convolution during reasoning, thereby improving computational efficiency. The EfficientRep structure and the RepBi-PAN structure make full use of the computing power of the hardware, significantly reduce the reasoning delay, and enhance the representation capability and multi-scale feature fusion capability.

[0079] like Figure 3 As shown, the backbone network Backbone of the present invention includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module, a fifth feature extraction module and a SimCSPSPPF module connected in sequence; wherein, the first feature extraction module includes 1 RepVGG Block, the second feature extraction module includes 1 RepVGGBlock and 1 fast-reparameterization module Faster-RepBlock, the third feature extraction module includes 1 RepVGG Block and 2 fast-reparameterization modules Faster-RepBlock, the fourth feature extraction module includes 1 RepVGG Block and 2 fast-reparameterization modules Faster-RepBlock, and the fifth feature extraction module includes 1 RepVGG Block and 1 fast-reparameterization module Faster-RepBlock.

[0080] On the P2 to P5 layers of the backbone network Backbone, 1, 2, 2, and 1 fast-reparameterization modules Faster-RepBlock are stacked respectively to replace the original reparameterization module RepBlock, taking into account both speed and accuracy.

[0081] In the backbone network, RepVGG Block is a downsampling convolution, which mainly extracts more semantic information by reducing the spatial dimension of the feature map. Figure 7 As shown in the figure, during the training phase, each RepVGGBlock includes a first branch, a second branch, a third branch and a ReLU activation function, and the first branch, the second branch and the third branch are connected to the ReLU activation function through an adder; the first branch and the second branch both include a convolution layer Conv and a batch normalization layer BN, and the third branch is a batch normalization layer BN. After training, the first branch, the second branch and the third branch of each RepVGG Block are simplified into a reparameterized convolution block RepConv using the reparameterization technique, thereby achieving efficient inference performance, as shown in the figure. Figure 8 shown.

[0082] The SimCSPSPPF module is used to enhance the representation capability of the model. Fig. 9 As shown, the SimCSPSPPF module includes a first convolution module, a second convolution module, a third convolution module, a first maximum pooling layer, a second maximum pooling layer, a third maximum pooling layer, a first splicing layer C, a fourth convolution module, a fifth convolution module, a second splicing layer C and a sixth convolution module connected in sequence; the third convolution module, the first maximum pooling layer, and the second maximum pooling layer are also connected to the first splicing layer, and the input end of the first convolution module is also connected to the second splicing layer through the seventh convolution module. The first convolution module contains 1 ConvModule, the second convolution module contains 3 ConvModules, the third convolution module contains 1 ConvModule, the fourth convolution module contains 1 ConvModule, the fifth convolution module contains 3 ConvModules, the sixth convolution module contains 1 ConvModule, and the seventh convolution module contains 1 ConvModule.

[0083] The RepBi-PAN structure adopted by the neck network Neck of the present invention integrates the feature maps of three adjacent layers through a bidirectional splicing module (BiC), which can better retain accurate positioning signals, especially important for the positioning of small objects. The neck network Neck is mainly composed of a convolutional layer Conv, a bidirectional splicing module BiC, a fast-reparameterization module Faster-RepBlock and a splicing module Concat. Figure 3The two Convs on the left are used for dimensionality reduction, the two Convs from bottom to top on the far right are downsampling convolutions, and Concat is responsible for splicing two feature maps of the same scale in the channel dimension.

[0084] Based on the consideration of accuracy priority, the present invention adopts a stacking structure of two fast-reparameterization modules Faster-RepBlock to replace the original reparameterization module RepBlock in the P3' to P5' layers of the neck network Neck. If the same purpose is achieved in other YOLO models, the corresponding number of stacks needs to be designed according to the model structure. The present invention adopts the method of replacing RepBlock with Faster-RepBlock, which not only ensures good feature extraction and feature fusion capabilities, but also ensures that the accuracy loss is within an acceptable range, while significantly improving the reasoning speed of the model, thereby improving the target detection efficiency.

[0085] The present invention designs a lightweight and efficient Faster-RepBlock to improve the original YOLOv6 model. Without sacrificing detection accuracy, it significantly reduces the number of model parameters and model scale, reduces computational complexity, and thus improves its deployment efficiency on edge devices with limited computing power, memory, and energy consumption, making the model more suitable for embedded edge devices with limited resources.

[0086] The present invention designs PRepConv to optimize the model structure, which significantly improves the model's reasoning speed and target detection efficiency. On edge devices, compared with the original YOLOv6 model, the present invention has no loss in detection accuracy, and the reasoning speed is increased by about 40%, which can better meet application scenarios with high real-time requirements, such as intelligent monitoring and unmanned driving.

[0087] The improved YOLOv6 model of the present invention significantly improves the reasoning speed and reduces the precision loss while maintaining good feature extraction and fusion capabilities. Compared with the existing lightweight solutions, the present invention avoids the trade-off between speed and precision through a more effective structural design, so that high-speed reasoning and high-precision detection can be achieved on edge devices.

[0088] The traditional YOLOv6 model uses the SIoU function (Simplified Intersection over Union) as the border loss function. The SIoU function takes into account the vector angle between the real box and the predicted box, and redefines the loss from the distance, angle, shape and IoU, which can better match the geometric characteristics of the real box and the predicted box. However, the SIoU function ignores the coverage area, center point distance, and consistency of the aspect ratio between the real box and the predicted box, which may cause the shape and size of the real box and the predicted box to not match, resulting in reduced accuracy of the target predicted box. In order to solve the above problems, the present invention improves the border regression loss function based on SIoU, and the improved border regression loss function is:

[0089] (1)

[0090] (2)

[0091] (3)

[0092] (4)

[0093] (5)

[0094] (6)

[0095] (7)

[0096] Among them, L represents the loss value, IoU represents the intersection-over-union ratio between the real box and the predicted box, represents the distance loss, Ω represents the shape loss, β 1 、β 2 represent the learnable weight coefficients, ρ represents the Euclidean distance between the center point of the real box and the predicted box, c represents the diagonal length of the minimum circumscribed rectangle of the predicted box and the real box, α represents the positive balance parameter, Indicates the consistency of the aspect ratio between the predicted box and the real box. , , , , Both represent intermediate quantities. , Represents the coordinates of the center point of the real box, , Represents the center point coordinates of the prediction box, , Represents the width and height of the minimum bounding rectangle of the predicted box and the real box respectively, θ represents the degree of attention to shape loss, , Respectively represent the width of the predicted box and the real box, , Represent the height of the predicted box and the true box respectively.

[0097] The improved border regression loss function of the present invention dynamically adjusts the influence of SIoU loss by introducing a learnable weight coefficient, so as to adaptively balance the various parts of the loss function in different training stages or different tasks. In addition, in order to further improve the regression accuracy of the target detection model, the improved border regression loss function also introduces constraints on the coverage area, center point distance, and consistency of the aspect ratio between the predicted box and the true box, ensuring that the shape of the predicted box can match the true box more accurately. The learnable weight coefficient provides more flexibility for the target detection model, and can automatically adjust the contribution ratio of each loss according to the learning goal of the current stage during the training process, thereby accelerating the convergence of the target detection model. The improved border regression loss function further optimizes the learning of the border shape. In the regression task, the improved border regression loss function not only focuses on the position and scale of the box, but also ensures the consistency of the aspect ratio between the predicted box and the true box, which helps to reduce the negative impact of shape mismatch on the prediction results, thereby significantly improving the regression accuracy of the target detection model.

[0098] Example 2

[0099] Although not shown, the embodiment of the present invention further provides a target detection method, comprising the following steps:

[0100] Step 1: Get the image to be detected;

[0101] Step 2: Obtain a trained target detection model using the target detection model training method in Example 1 of the present application;

[0102] Step 3: Use the trained target detection model to perform target detection on the image to be detected.

[0103] Example 3

[0104] An embodiment of the present invention also provides an electronic device, which includes: a memory, a processor, and a computer program / instructions stored in the memory, and the processor executes the computer program / instructions to implement the target detection model training method or target detection method in the embodiment of the present application.

[0105] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes according to the programs and / or data stored in the read-only memory (ROM) or the programs and / or data loaded from the storage part into the random access memory (RAM). The processor can be a multi-core processor or can include multiple processors. In some embodiments, the processor can include a general-purpose main processor and one or more special coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM, various programs and data required for device operation are also stored. The processor, ROM, and RAM are connected to each other via a bus. The input / output (I / O) interface is also connected to the bus.

[0106] The processor and memory are used together to execute the program / instructions stored in the memory. When the program / instructions are executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0107] Although not shown, an embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the target detection model training method or target detection method in the embodiment of the present application.

[0108] Readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0109] Although not shown, an embodiment of the present invention also provides a computer program product, including: a computer program / instruction, which, when executed by a processor, implements the target detection model training method or target detection method in the embodiment of the present application.

[0110] What is disclosed above is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or modifications within the technical scope disclosed in the present invention, which should be covered within the protection scope of the present invention.

Claims

1. A target detection model training method, characterized in that: The training method comprises: Obtain a sample data set; wherein the sample data set includes a plurality of labeled training images, and the labels are real frames of targets in the training images; Constructing a target detection model; wherein the target detection model is an improved YOLOv6 model, the improved YOLOv6 model includes a backbone network, a neck network and a detection network, and on the basis of the original YOLOv6 model, a fast-reparameterization module is used to replace the reparameterization modules in the backbone network and the neck network; the fast-reparameterization module includes a partially reparameterized convolution block, two convolution modules and an addition fusion module connected in sequence, and the addition fusion module is also connected to the input end of the partially reparameterized convolution block through a jump connection; the partially reparameterized convolution block includes 1 RepVGG Block, 1 Identity module and 1 splicing module, the RepVGG Block is used to perform a reparameterized convolution operation on 1 / 2 channel features, the Identity module is used to directly output the remaining 1 / 2 channel features, and the outputs of the RepVGGBlock and Identity modules are merged into a final output through the splicing operation of the splicing module; Using the sample data set to train the target detection model to obtain a trained target detection model; The target detection model is trained using the sample data set, including: Inputting the training image into the target detection model, and using the backbone network to extract features of the training image to obtain feature maps P2, P3, P4, and P5 of different scales; The neck network is used to fuse low-level semantic features and high-level semantic features in feature maps P2, P3, P4, and P5 to obtain fused feature maps P3', P4', and P5' of different scales; Using the detection network to perform target detection on the fused feature map P3', the fused feature map P4' and the fused feature map P5', respectively, to obtain prediction frames of different sizes; A loss value between the predicted box and the corresponding real box is calculated, and the parameters of the target detection model are adjusted according to the loss value.

2. The target detection model training method according to claim 1, characterized in that: Each of the convolution modules includes a convolution layer, a batch normalization layer and an activation function connected in sequence.

3. The target detection model training method according to claim 1, characterized in that: The backbone network adopts the EfficientRep structure, and the neck network adopts the RepBi-PAN structure.

4. The target detection model training method according to claim 3, characterized in that: The backbone network includes a first feature extraction module, a second feature extraction module, a third feature extraction module, a fourth feature extraction module, a fifth feature extraction module and a SimCSPSPPF module connected in sequence; wherein the first feature extraction module includes 1 RepVGG Block, the second feature extraction module includes 1 RepVGG Block and 1 fast-reparameterization module, the third feature extraction module includes 1 RepVGG Block and 2 fast-reparameterization modules, the fourth feature extraction module includes 1 RepVGG Block and 2 fast-reparameterization modules, and the fifth feature extraction module includes 1 RepVGG Block and 1 fast-reparameterization module; The first feature extraction module is used to extract features from the training image to obtain a feature map P1; the second feature extraction module is used to extract features from the feature map P1 to obtain a feature map P2; the third feature extraction module is used to extract features from the feature map P2 to obtain a feature map P3; the fourth feature extraction module is used to extract features from the feature map P3 to obtain a feature map P4; the fifth feature extraction module is used to extract features from the feature map P4 to obtain a feature map P5; In the neck network, two fast-reparameterization modules connected in sequence are used to replace the original reparameterization module.

5. The target detection model training method according to claim 4, characterized in that: In the training stage, each of the RepVGG Blocks includes a first branch, a second branch, a third branch and a ReLU activation function, wherein the first branch, the second branch and the third branch are connected to the ReLU activation function through an adder; the first branch and the second branch both include a convolution layer and a batch normalization layer, and the third branch is a batch normalization layer; After training, the reparameterization technique is used to simplify the first branch, second branch, and third branch of each RepVGG Block into a reparameterized convolutional block for efficient inference.

6. The target detection model training method according to claim 4, characterized in that: The SimCSPSPPF module includes a first convolution module, a second convolution module, a third convolution module, a first maximum pooling layer, a second maximum pooling layer, a third maximum pooling layer, a first splicing layer, a fourth convolution module, a fifth convolution module, a second splicing layer and a sixth convolution module, which are connected in sequence; the third convolution module, the first maximum pooling layer, the second maximum pooling layer are also connected to the first splicing layer, and the input end of the first convolution module is also connected to the second splicing layer through the seventh convolution module.

7. The target detection model training method according to claim 1, characterized in that: The improved border regression loss function is used to calculate the loss value between the predicted box and the corresponding real box; the improved border regression loss function is: ； ； ； ； ； ； ； Among them, L represents the loss value, IoU represents the intersection-over-union ratio between the real box and the predicted box, represents the distance loss, Ω represents the shape loss, β 1 、β 2 represent the learnable weight coefficients, ρ represents the Euclidean distance between the center point of the real box and the predicted box, c represents the diagonal length of the minimum circumscribed rectangle of the predicted box and the real box, α represents the positive balance parameter, Indicates the consistency of the aspect ratio between the predicted box and the real box, , , , , Both represent intermediate quantities. , Represents the coordinates of the center point of the real box, , Represents the center point coordinates of the prediction box, , Represents the width and height of the minimum bounding rectangle of the predicted box and the real box respectively, θ represents the degree of attention to shape loss, , Respectively represent the width of the predicted box and the real box, , Represent the height of the predicted box and the true box respectively.

8. A target detection method, characterized in that: The detection method comprises: Acquire the image to be detected; Obtaining a trained target detection model using the target detection model training method according to any one of claims 1 to 7; The trained target detection model is used to perform target detection on the image to be detected.

9. An electronic device comprising a memory, a processor, and a computer program / instruction stored in the memory, characterized in that: The processor executes the computer program / instructions to implement the target detection model training method according to any one of claims 1 to 7 or the target detection method according to claim 8.

10. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, it implements the target detection model training method as described in any one of claims 1 to 7 or the target detection method as described in claim 8.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, it implements the target detection model training method as described in any one of claims 1 to 7 or the target detection method as described in claim 8.

Images