Incremental target detection method and system based on YOLOv5 and EWC

Abstract

The invention discloses an incremental target detection method and system based on YOLOv5 and EWC. The detection method comprises the following steps: S1, initializing a YOLOv5 network model by using a first data set to obtain a base model; s2, acquiring a pseudo tag in the new data set by using the base model, and updating the new data set to obtain a second data set; s3, training the second data set by adopting a regularization loss function and adjusting parameters of the base model to obtain a first model; and S4, updating the parameter weight of the first model, recording the first model as a base model, and repeating the steps S2-S3 for training until the first model after parameter adjustment meets a training ending condition, thereby obtaining a network model for incremental target detection. According to the method, the YOLOv5 network model is combined with the EWC algorithm, the high accuracy is kept, the detection and calculation speed is very high, the performance on training data is good, the method can be well generalized to new and unseen new data, and the accuracy and robustness of target detection are improved.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to an incremental target detection method and system based on YOLOv5 and EWC. Background Art

[0002] Object detection is to find all the objects of interest in the image and determine their categories and locations. Currently, object detection has been widely used in schools, factories, commerce, military and other fields, and plays a key role in them. The current two-stage object detection model and single-stage object detection model can almost cope with the challenges of all large-scale data sets on the market. Although the modern object detection technology based on deep convolutional neural network can accurately and quickly realize the positioning and classification tasks of the target area, and the learning ability is also amazing, a target detection network model can learn thousands of categories, but almost all current target detection models can only recognize the target categories in the trained data set. When encountering "strange" targets, they cannot recognize them, which is very different from the "lifelong learning" of humans and other primates. Therefore, for the "artificial intelligence" technology of the target detection model, it is not only required that the target detection network model can complete the detection of existing data, but also that the target detection network model can continue to learn, so as to achieve the effect of not only detecting the original categories learned, but also continuously learning new categories, that is, to achieve incremental target detection.

[0003] Incremental target detection requires the model to gradually learn new knowledge in a constantly changing environment without forgetting old knowledge. It is suitable for target detection tasks that need to be continuously updated to adapt to new objects or scenes. Currently, the direct use of two-stage target detection technology and single-stage target detection technology for incremental detection will lead to the occurrence of the "catastrophic forgetting" problem. Incremental target detection, that is, incremental learning, is exactly the algorithm to solve the "catastrophic forgetting" that occurs in the model during the incremental learning process. Combining the target detection algorithm with the incremental learning algorithm can achieve incremental target detection. Patent document CN110837856B discloses a neural network training method, which includes: obtaining an initial neural network and a sample image containing a new category target, the initial neural network represents a neural network for detecting at least one category of old category targets, and the new category target does not belong to at least one category of old category targets; obtaining annotation information of the new category target in the sample image; using the initial neural network to perform target detection on the sample image to obtain at least one target detection result of the sample image, and when there is an old category target in the sample image, using the at least one target detection result as the annotation information of the old category target in the sample image; the annotation information includes an annotation box; in response to the situation that the overlapping area of ​​the annotation box of any old category target and the annotation box of a new category target is greater than or equal to a first area threshold, deleting the corresponding annotation information of the old category target; in response to the situation that the overlapping area of ​​the annotation box of any old category target and the annotation box of each new category target is less than the first area threshold, retaining the corresponding annotation information of the old category target; based on the annotation information of the new category target and the annotation information of the old category target, incremental training is performed on the basis of the initial neural network to obtain an object detection network for detecting new category targets and old category targets.

[0004] The above methods provide a neural network that can detect both new and old category targets at the same time. However, in the specific application process, it is impossible to select a suitable initial neural network, which greatly increases the parameter adjustment and calculation amount. In addition, the addition of inappropriate regularization terms will cause the model to overfit on the training data, with poor generalization ability, and unable to effectively control the complexity of the model, and the training time is long. Therefore, it is necessary to develop a lightweight incremental target detection method that performs well on training data. Summary of the invention

[0005] The purpose of the present invention is to overcome the problems that the initial neural network and regularization term selected in the prior art are inappropriate, resulting in increased parameter adjustment and calculation, overfitting on training data, poor generalization ability, and inability to effectively control the complexity of the model, and to provide an incremental target detection method and system based on YOLOv5 and EWC.

[0006] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0007] An incremental target detection method based on YOLOv5 and EWC, comprising the following steps:

[0008] S1. Use the first data set to initialize the YOLOv5 network model to obtain the base model

[0009] S2. Using the base model Acquire pseudo labels in a new data set and update the new data set to obtain a second data set;

[0010] S3, inputting the second data set into the base model with the EWC algorithm added In the example, the regularized loss function is used to train the second data set and adjust the parameters of the base model to obtain the first model.

[0011] S4. Update the first model The parameter weights of the first model are As the base model Repeat steps S2 to S3 to train until the first model after parameter adjustment is When the training end conditions are met, a network model for incremental target detection is obtained, and then incremental target detection is performed.

[0012] In the above technical solution, the first data set is first used to train the YOLOv5 network model to obtain a base model, and then the base model is used to obtain pseudo labels in the second data set containing new knowledge, and the new data set is updated in combination with the pseudo labels to obtain the second data set; the second data set is input into the base model with the EWC (elastic weight consolidation) algorithm added, and the second data set is trained with a regularized loss function and the model parameters are fine-tuned to obtain the first model, in preparation for learning again, the parameter weights of the first model are updated according to the above, and then recorded t←t+1, repeat steps S2~S3 for training to obtain a network model for incremental target detection. This is a method with positive significance for the research on the "lifelong learning" direction of target detection.

[0013] Through the above technical method, the present invention integrates the loss functions of the two algorithms YOLOv5 and EWC for incremental target detection. The key to the integration is to add the EWC algorithm as a regularization term in the YOLOv5 loss function. The EWC algorithm uses the loss function to perform incremental learning through posterior information, that is, by constraining the parameters important to the tasks learned by the entire network model, so that it remains close to its old value after learning the new task to achieve incremental learning, and adding pseudo-label processing in the experiment effectively solves the problem of catastrophic forgetting. At the same time, the YOLOv5 network model used in the present invention adopts a lightweight model design. While maintaining a high accuracy rate, the detection and calculation speeds are very fast. It is an efficient target detection network. After combining the EWC algorithm, it performs well on training data and can be well generalized to new, unseen new data, improving the accuracy and robustness of target detection.

[0014] As a preferred solution of the present invention, the YOLOv5 network model includes an input end, a backbone network Backbone, Neck and a detection layer Prediction, wherein:

[0015] The input end is used to process input data;

[0016] The backbone network Backbone includes a Focus module, a CBL module, a CSP module and an SPP module;

[0017] The Neck module has the FPN+NewCSP-PAN structure;

[0018] The detection layer Prediction is used to calculate the loss function.

[0019] As a preferred embodiment of the present invention, step S1 specifically comprises the following steps:

[0020] S11, import the YOLOv5 network model to be initialized;

[0021] S12, input the first data set to be trained to initialize the YOLOv5 network model;

[0022] S13. After the data set learning is completed, the initialized base model is obtained.

[0023] As a preferred solution of the present invention, the parameter weights of the YOLOv5 network model initialized in step S11 are all set to 1.

[0024] As a preferred solution of the present invention, the specific steps of step S2 are: inputting a new data set, wherein the new data set includes new data that does not belong to the first data set, using the base model to predict pseudo labels of targets in the first data set that are related to the first data set, wherein the pseudo labels include category information and location information of the targets; and fusing the pseudo labels with the labels of the new data set as the second data set.

[0025] As a preferred embodiment of the present invention, the regularized loss function is as follows:

[0026]

[0027] Where loss(object) is the loss function of YOLOv5, λ is a hyperparameter, is the weight importance of parameter i in the tth task, θ i represents the actual value of the i-th parameter, represents the optimal parameters obtained after training for the tth task.

[0028] As a preferred solution of the present invention, the specific method of adjusting the parameters of the base model is:

[0029] S311 and EWC use Gaussian posteriors for each model parameter and obtain the Hessian matrix through multivariate Gaussian distribution and Laplace approximation;

[0030] S312, estimate the diagonal structure of the Hessian to calculate the diagonal structure of each parameter θ i To break it down:

[0031]

[0032] in Denotes the Hessian matrix H t The diagonal matrix of

[0033] S313. Calculate the minimum loss function from the first task training to the t+1th task:

[0034]

[0035] Among them, λ is a hyperparameter used to balance the elasticity and stability of the model.

[0036] As a preferred solution of the present invention, the specific method of step S311 is:

[0037] Set the first data set D 1:t Trained parameters Using the second dataset D t+1 Calculate the optimal parameters of the new model Maximize the posterior information of all learned data sets:

[0038] p(θ|D 1:t+1 )∝p(D t+1 |θ)p(θ|D 1:t )

[0039] Among them, p(D t+1 |θ) is the likelihood value of the second data set, p(θ|D 1:t ) is the posterior probability;

[0040] The posterior probability in S311 and the optimal parameters trained by the first data set are calculated by multivariate Gaussian distribution. The optimal parameters are expressed as:

[0041]

[0042] in, = -logp(θ|D 1:t )exist The Hessian matrix at ;

[0043] Use Laplace approximation to calculate the Hessian matrix and the posterior probability p(θ|D 1:t+1 ):

[0044]

[0045] Combined, we get the recursive expression of the Hessian matrix:

[0046]

[0047] As a preferred solution of the present invention, the Hessian diagonal matrix can be calculated by the following recursive equation:

[0048]

[0049] As a preferred solution of the present invention, the parameter weight is updated recursively, and the formula is:

[0050]

[0051] in represents the loss of the jth sample on the tth task, Represents the importance of all parameters learned until task t+1; for the first task, initialize the importance of parameters, let

[0052] As a preferred solution of the present invention, the specific implementation method of step S4 is: according to t=0,1,2,..Update model The weight of each parameter, then record Repeat the training t←t+1 until the first model with adjusted parameters is The training end conditions are met and a network model for incremental target detection is obtained.

[0053] Another aspect of the present invention provides an incremental target detection system based on YOLOv5 and EWC, the system comprising:

[0054] A base model module, used to initialize the YOLOv5 network model using the first data set to obtain a base model;

[0055] A new data acquisition module, used to use the base model to acquire pseudo labels in a new data set and update the new data set to obtain a second data set;

[0056] A first model module, used for inputting the second data set into the base model with the EWC algorithm added, training the second data set using a regularized loss function and adjusting the parameters of the base model to obtain a first model;

[0057] The incremental target detection network model module is used to update the parameter weights of the first model, record the first model as the base model, repeat steps S2 to S3 for training until the first model with adjusted parameters meets the training end conditions, and obtain a network model for incremental target detection.

[0058] The present invention also provides an electronic device, comprising at least one processor and a memory communicatively connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned incremental target detection method based on YOLOv5 and EWC.

[0059] The present invention also provides a computer-readable storage medium, wherein the readable storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor to implement the above-mentioned incremental target detection method based on YOLOv5 and EWC.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] 1. The invention integrates the loss functions of the two algorithms YOLOv5 and EWC for incremental target detection. The key to the integration is to add the EWC algorithm as a regularization term in the YOLOv5 loss function. The EWC algorithm uses the loss function to perform incremental learning through posterior information, that is, by constraining the parameters important to the tasks learned by the entire network model, so that it remains close to its old value after learning the new task to achieve incremental learning, and adding pseudo-label processing in the experiment effectively solves the problem of catastrophic forgetting. At the same time, the YOLOv5 network model used in the invention adopts a lightweight model design. While maintaining a high accuracy rate, the detection and calculation speeds are very fast. It is an efficient target detection network. After combining with the EWC algorithm, it performs well on training data and can be well generalized to new and unseen new data, improving the accuracy and robustness of target detection.

[0062] 2. The method of the present invention can be easily applied to different target detection tasks and data sets, and can be applied to intelligent monitoring systems, autonomous driving systems, industrial automation and other fields, and has good scalability and adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 It is a flow chart of the incremental target detection method of YOLOv5 and EWC of the present invention;

[0064] Figure 2 This is the structure diagram of the YOLOv5 network model;

[0065] Figure 3 This is the Focus module diagram of the YOLOv5 network model structure;

[0066] Figure 4 This is the CBL module diagram of the YOLOv5 network model structure;

[0067] Figure 5 This is the CSP module diagram of the YOLOv5 network model structure;

[0068] Figure 6 This is the SPP module diagram of the YOLOv5 network model structure;

[0069] Figure 7 This is the FPA structure diagram of the YOLOv5 network model structure;

[0070] Figure 8 Schematic diagram of DIOU calculation for the YOLOv5 network model structure. DETAILED DESCRIPTION

[0071] In order to more clearly describe the invention purpose, technical scheme and technical effect advantages in the specific implementation case of the present invention, the scheme in the specific embodiment will be described in detail in combination with the drawings of the specification of the present invention. The specific technical scheme involved in the following specific embodiments is only for a clear and complete description of the innovative technical scheme of the present invention. It is only a part of the specific implementation scheme that can be adopted by the present invention, not all embodiments, and should not be understood as a limitation on the innovative scheme of the present invention. Any scheme adopting the same inventive concept of the present invention should be included in the protection scope of the present invention.

[0072] Secondly, the description of the drawings in the specific embodiments of the present invention is only for the convenience of technical personnel to understand the solution of the present invention. The partial details in the drawings are for the convenience of clearly presenting the technical solution. It should not be considered that all technical features in the drawings must be included in the specific implementation cases, and the detailed features in the drawings should not be identified as additional limitations on the innovative technical solution of the present invention. The components in the various embodiments described and shown in the drawings can be combined and arranged according to different configurations, and these changes in combination and arrangement should be identified as part of the entire embodiment of the innovative solution of the present invention and included in the scope of protection of the present invention.

[0073] In summary, the schemes or descriptions presented in the specific embodiments and drawings of the present invention are not intended to limit the scope of protection claimed, but are merely selected embodiments / cases to help technicians understand the relevant innovative solutions. Based on these embodiments, all other equivalent or parallel embodiments obtained by those skilled in the art without creative work are within the scope of protection claimed by the present invention.

[0074] Example 1

[0075] like Figure 1 , this embodiment provides an incremental target detection method based on YOLOv5 and EWC, comprising the following steps:

[0076] S1. Use the first data set to initialize the YOLOv5 network model to obtain the base model

[0077] S2. Using the base model Obtain the pseudo labels in the new data set and update the new data set to obtain the second data set;

[0078] S3. Input the second data set into the base model with the EWC algorithm added In the example, the regularized loss function is used to train the second data set and adjust the parameters of the base model to obtain the first model.

[0079] S4. Update the first model The parameter weights of the first model As a base model t←t+1, repeat steps S2 to S3 for training until the first model after parameter adjustment When the training end conditions are met, a network model for incremental target detection is obtained, and incremental target detection is performed.

[0080] like Figure 2 As shown in the figure, the YOLOv5 network model includes an input end, a backbone network Backbone, Neck and a detection layer Prediction, wherein the input end is used to process the input data; the backbone network Backbone includes a Focus module, a CBL module, a CSP module and an SPP module; the Neck module is an FPN+NewCSP-PAN structure; and the detection layer Prediction is used to calculate the loss function.

[0081] The input end processes input data including Mosaic data enhancement, adaptive anchor box calculation and adaptive image scaling. Mosaic data enhancement uses at least two images to be spliced ​​by random scaling, random cropping and random arrangement, for example, 4 images. The processing method of Mosaic data enhancement can enrich the detection data set, enhance robustness through random scaling, and save computing costs by requiring a smaller batch for training. The processing method of adaptive anchor box calculation compares the predicted box with the actual box to calculate the difference and then updates it in reverse, thereby continuously optimizing the model parameters; the processing method of adaptive image scaling is to adaptively add the least black edge to the original image to reduce calculation and improve the inference speed.

[0082] In the backbone network Backbone, the Focus module is a special downsampling method; Figure 3 , The Focus module is used to slice and splice images. During slicing, the pixel values ​​of each input image are taken at intervals, so that one image is turned into at least two images, for example, four images. The multiple images have high similarity and complement each other, which not only ensures that information is not lost, but also integrates W and H information into the channel space, expands the input channel to multiple times of the original, and then sends the obtained image to the convolution layer, so that the multi-fold down-sampling feature map can be obtained without losing information.

[0083] The CBL module is a standard convolution module, such as Figure 4 The CBL module includes convolution CONV, normalization processing BN, and activation function LeakyReLU, which is a three-part structure for data processing.

[0084] The CSP module is used to process the output of the CBL module, such as Figure 5, the CSP module includes CSP1_1 module and CSP2_1 module. The CSP1_1 module uses a CBL unit on the main branch, then connects several residual components, concatenates with the residual branch after convolution, and finally performs normalization processing BN, activation function LeakyReLU and CBL unit processing; compared with CSP1_1, the CSP2_1 module replaces the residual component with the CBL unit, and the rest of the structure is the same as CSP1_1.

[0085] The SPP module is the processing module of the CBL unit, such as Figure 6 ,The SPP module is a CBL unit that connects three serial maximum pooling layers, which are then concatenated with each other, and finally connected to another CBL unit.

[0086] In the Neck module, FPN is a top-down high-level semantic information transmission method, which is used to enhance information; the NewCSP-PAN structure is a bottom-up low-level strong positioning information transmission method, which is used to solve the defect of FPN not transmitting position information. Figure 7 ,The FPN+NewCSP-PAN structure in the Neck module combines top-down and bottom-up feature fusion.

[0087] The detection layer is a loss function, and NMS non-maximum suppression is used. The rectangular box loss is calculated using the CIOU Loss function, and the confidence loss and classification loss are calculated using the BCE Loss function. The calculation formula of CIOU is:

[0088]

[0089] Among them, CIOU is the similarity of shape, IOU is the ratio of the intersection and union of two bounding boxes, ρ is the center point distance between the prediction box A and the target box B, c is the diagonal length of the minimum rectangle surrounding the two boxes, DIOU is an indicator to measure the similarity of the two bounding boxes, and the concept of center point distance is introduced on the basis of IOU. v is used to measure the similarity of the aspect ratio of the two boxes, and α is the influencing factor representing v, then:

[0090]

[0091] w l 、h l and w p 、h p are the width and height of the two bounding boxes respectively.

[0092] like Figure 8 , DIOU calculation formula is:

[0093]

[0094] ρ 2 =(x p -x1) 2 +(y p -y l ) 2

[0095] c 2 =(max(x p2 ,x l2 )-min(x p1 ,x l1 )) 2 +(max(y p2 ,y l2 )-min(y p1 ,y l1 )) 2

[0096] The calculation formula of the BCELoss function is as follows:

[0097] Loss BCE (z,x,y)

[0098] =-L(z,x,y)*logP(z,x,y)-(1-L(z,x,y))*log(1-P(z,x,y)),0≤z<3,0≤x<80,0≤y<80

[0099] Where L(z,x,y) is the confidence label matrix, and P(z,x,y) is the predicted confidence matrix of the predicted box. In order to obtain L(z,x,y), the mask matrix and the CIOU values ​​of the predicted box and the target box are needed. For the grids where the mask matrix is ​​True, its CIOU needs to be calculated, and the confidence label is the calculated CIOU value; for the grids where the mask matrix is ​​False, its confidence label is directly assigned to 0, thus obtaining a 3*80*80 confidence label matrix.

[0100] The calculation steps of the mask matrix in the calculation method are:

[0101] S61 . Divide an input image of 640*640 or 480*480 into one of three grid sizes: 80*80, 40*40, and 20*20.

[0102] S62, parse the center coordinates (x, y), height and width (w, h) of the target box marked in the json file of each image, and the actual height and width (w i ,h i ).

[0103] S63, according to (x', y') obtained in S62, calculate the grid coordinates (xg ,y g ).

[0104] S64, S63 grid coordinates (x g ,y g ) is rounded down to get the coordinate (x0, y0). The left / right and top / bottom of the coordinate (x0, y0) are taken into account and recorded as (x1, y0) and (x0, y1) respectively. The calculation method of x1 and y1 is:

[0105] x1=x0+(2*round(x g -x0)-1)

[0106] y1=y0+(2*round(y g -y0)-1

[0107] S65. Assign three anchors to the grids where the coordinates (x0, y0), (x1, y0) and (x0, y1) are located, denoted as anchor0, anchor1 and anchor2.

[0108] S66. Calculate (w gt ,h gt ) to (w0,h0), (w1,h1), (w2,h2), where (w0,h0), (w1,h1), (w2,h2) are the width and height of S65anchor0,anchor1,anchor2 respectively;

[0109] S67. Assign values ​​to the mask matrix by combining the three coordinates (x0, y0), (x1, y0), (x0, y1) in S64 and the three values ​​(m0, m1, m2) calculated in e) to obtain the mask matrix:

[0110]

[0111] The specific steps of step S1 are:

[0112] S11, importing the YOLOv5 network model to be initialized; in this embodiment, the parameter weights of the initialized YOLOv5 network model are all set to 1;

[0113] S12, input the first data set to be trained to initialize the YOLOv5 network model;

[0114] S13. After the data set learning is completed, the initialized base model is obtained.

[0115] Pseudo-labeling is to mark the objects in the old dataset that have not been assigned labels in the new dataset. In order to achieve pseudo-labeling, before learning a new dataset each time, the new dataset is first sent to the base model trained with the original first dataset for prediction, so as to obtain the label of the old dataset that was originally learned in the new dataset, and then the label information is merged with the label information of the new dataset. Finally, the new dataset and the fused label information are input into the network model together, so that the network model can learn the new dataset. The specific steps of step S2 are: input a new dataset, the new dataset includes data that does not belong to the first dataset, and the base model is used to predict the pseudo-label of the target related to the first dataset in the new dataset. The pseudo-label includes the category information and location information of the target; the pseudo-label is merged with the label of the new dataset as the second dataset.

[0116] Input the second data set to the base model adding the EWC algorithm In the example, the regularized loss function is used to train the second data set, and the parameters of the base model are fine-tuned to obtain the first model. In this technical solution, incremental target detection is achieved by fusing the loss functions of the YOLOv5 and EWC algorithms. The key to the fusion is to add the EWC algorithm as a regularization term in the YOLOv5 network model loss function and record the information of all learned data, thereby achieving incremental target detection. Therefore, the incremental target detection loss function can be expressed as:

[0117]

[0118] The regularized loss function is expressed as:

[0119]

[0120] Where loss(object) is the loss function of the YOLOv5 network model. is the weight importance of parameter i in the tth task, θ i is the actual value of the i-th parameter, is the optimal parameter obtained after training the tth task. Therefore, the more specific expression of the regularized loss function is:

[0121]

[0122] λ is a hyperparameter used to balance the elasticity and stability of the model; the regularized loss function of the target object depends not only on the likelihood function of the second data set, but also on the importance of each parameter of the trained model, and the importance of each parameter depends on the inverse of all learned data set parameters.

[0123] The loss function of the YOLOv5 network model is divided into two parts, namely the classification loss for identifying the target category and the regression loss for locating the target position. The loss function of the YOLOv5 network model is further subdivided, and the loss is divided into three categories: rectangular box loss, confidence loss, and classification loss. The rectangular box loss measures the loss between the predicted box and the target box, and CIOU is used to calculate it in YOLOv5; the confidence loss measures the reliability of each predicted box; and the classification loss measures the accuracy of object classification. Both the confidence loss and the classification loss YOLOv5 use BCE Loss to calculate.

[0124] It can be seen that the key to achieving incremental target detection in this method is to add a regularization term to the original YOLOv5 loss function. The regularization term records the information of all learned data, and then iterates the parameters through the stochastic gradient descent method to find the optimal parameters for each training data set. The EWC algorithm uses posterior information for incremental learning, that is, it constrains the parameters that are important to the tasks learned by the entire network model so that they remain close to their old values ​​after learning new tasks. The EWC algorithm uses Gaussian posteriors for each model parameter, and obtains its approximate value through multivariate Gaussian distribution and Laplace approximation.

[0125] In the process of adjusting the parameters of the base model, the EWC algorithm uses Gaussian posteriori for each parameter and calculates the Hessian diagonal matrix at the end of each task training. The calculated Hessian diagonal matrix can be considered as a regularized fine-tuning of the parameters on the second data set, so that the parameters important to the old task do not change much. The importance measure of the parameters is determined by the likelihood function of each training stage, using the following steps:

[0126] Set the first data set D 1:t Trained parameters Using the second dataset D t+1 Calculate the optimal parameters of the new model Maximize the posterior information of all learned data sets:

[0127] p(θ|D 1:t+1 )∝p(D t+1 |θ)p(θ|D 1:t )

[0128] Where p(D t+1 |θ) is the likelihood value of the second data set, p(θ|D 1:t ) is the posterior probability;

[0129] The above posterior probability and the optimal parameters trained by the first data set are calculated through multivariate Gaussian distribution, so the optimal parameters trained by the first data set can be expressed as:

[0130]

[0131] is -logp(θ|D 1:t )exist The Hessian matrix at .

[0132] Using Laplace approximation to calculate the Hessian matrix and the posterior probability p(θ|D 1:t+1 ):

[0133]

[0134] Combining the above results, we get the recursive expression of the Hessian matrix:

[0135]

[0136] After obtaining the Hessian matrix, the Hessian diagonal matrix needs to be calculated at the end of each task training. Since the size of the Hessian matrix is ​​the same as the number of parameters in the entire model, it cannot be calculated accurately. The EWC algorithm estimates the diagonal structure of the Hessian for each parameter θ i To break it down:

[0137]

[0138] in, Denotes the Hessian matrix H t The calculated Hessian diagonal matrix can be considered as a regularized fine-tuning of the parameters on the second dataset, so that the parameters important to the old task do not change much, and the importance measure of the parameters is determined by the likelihood function of each training stage.

[0139] Therefore, for training from the first task to the t+1th task, the minimized loss function should be written as:

[0140]

[0141] Among them, λ is a hyperparameter used to balance the elasticity and stability of the model, and the Hessian diagonal matrix can be calculated by the following recursive equation:

[0142]

[0143] in Denotes the parameter θ i The second-order partial derivative of .

[0144] Therefore, in order to obtain the optimal parameters Directly use the second dataset D t+1 For the first model parameter Fine-tune the model parameters. After fine-tuning, the importance of each parameter needs to be calculated. It is worth noting that the importance of each parameter on task t+1 is calculated using an additional epoch. At this time, the gradient is not updated in each iteration. The calculated parameter importance is used express. Represents the importance of all parameters learned until task t+1, and is used to prepare for the learning of the next task.

[0145] The specific implementation method of step S4 is: according to t=0,1,2,..Update model The weight of each parameter, then record Repeat the training for t←t+1, where represents the loss of the jth sample on the tth task, Represents the importance of all parameters learned until task t+1, and is used to prepare for the learning of the next task. In each repeated training, the new base model is used to obtain the pseudo labels in the new data set and update the new data set, and then the updated data set is input into the base model with the EWC algorithm added for training to adjust the parameters of the model. Repeat this process until the first model with adjusted parameters meets the training end condition. The training end condition can be set according to the actual situation, such as reaching a certain number of training times, loss function convergence, etc. Finally, a network model for incremental target detection is obtained.

[0146] Example 2

[0147] This embodiment provides an incremental target detection system based on YOLOv5 and EWC, and the system includes:

[0148] A base model module, used to initialize the YOLOv5 network model using the first data set to obtain a base model;

[0149] A new data acquisition module, used to obtain pseudo labels in a new data set using a base model and update the new data set to obtain a second data set;

[0150] A first model module, used for inputting the second data set into the base model with the EWC algorithm added, training the second data set using a regularized loss function and adjusting the parameters of the base model to obtain a first model;

[0151] The incremental target detection network model module is used to update the parameter weights of the first model. The first model is taken as the base model, t←t+1, and steps S2 to S3 are repeated for training until the first model after parameter adjustment meets the training end condition, thereby obtaining a network model for incremental target detection.

[0152] The systems or modules described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described in various modules according to their functions. Of course, when implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware, or the module that implements the same function can be implemented by a combination of multiple sub-modules.

[0153] This embodiment also provides an electronic device, including at least one processor, a memory connected to the at least one processor, and at least one input / output interface connected to the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the incremental target detection method based on YOLOv5 and EWC in the aforementioned embodiment 1. The input / output interface may include a display, a keyboard, a mouse, and a USB interface for inputting and outputting data.

[0154] The electronic device can be an electronic device for a client, such as a mobile phone, a laptop computer, a tablet computer, a desktop computer, etc., to execute an incremental target detection method based on YOLOv5 and EWC in Example 1.

[0155] Those skilled in the art can understand that: all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), disks or optical disks, etc. Various media that can store program codes.

[0156] When the above-mentioned integrated unit of the present invention is implemented in the form of a software functional unit and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0157] This embodiment also provides a computer-readable storage medium, which stores at least one instruction, at least one program, code set or instruction set. The at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the above-mentioned incremental target detection method based on YOLOv5 and EWC.

[0158] Computer readable storage media can be tangible devices that hold and store instructions used by instruction execution devices. Computer readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any combination thereof.

[0159] For those skilled in the art, when understanding the solutions described in the specific embodiments of the present invention, they can refer to the conventional technical manuals in the field. At the same time, for the places where the above-mentioned terms appear, they can make appropriate understandings or adjustments for reference, and deduce the implementation of the same or similar technical solutions without paying any creative work.

[0160] The above embodiments describe only the basic principles, main features and / or advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and the invention content of the specification only describe the principles or specific cases of the present invention. Without departing from the essence of the innovative idea of ​​the present invention, the innovative scheme of the present invention may be subject to various changes and improvements, and these changes and improvements all fall within the scope of protection required by the present invention.

Claims

1. An incremental target detection method based on YOLOv5 and EWC, characterized in that: The following steps are involved: S1. Use the first data set to initialize the YOLOv5 network model to obtain the base model S2. Using the base model Acquire pseudo labels in a new data set and update the new data set to obtain a second data set; S3, inputting the second data set into the base model with the EWC algorithm added In the example, the regularized loss function is used to train the second data set and adjust the parameters of the base model to obtain the first model. S4. Update the first model The parameter weights of the first model are As the base model Repeat steps S2 to S3 to train until the first model after parameter adjustment is The training end conditions are met and a network model for incremental target detection is obtained.

2. The incremental target detection method based on YOLOv5 and EWC according to claim 1, characterized in that: The YOLOv5 network model includes an input end, a backbone network Backbone, Neck and a detection layer Prediction, wherein: The input end is used to process input data; The backbone network Backbone includes a Focus module, a CBL module, a CSP module and an SPP module; The Neck module has the FPN+NewCSP-PAN structure; The detection layer Prediction is used to calculate the loss function.

3. The incremental target detection method based on YOLOv5 and EWC according to claim 1, characterized in that: The specific steps of step S1 are: S11, import the YOLOv5 network model to be initialized; S12, input the first data set to be trained to initialize the YOLOv5 network model; S13. After the data set learning is completed, the initialized base model is obtained.

4. The incremental target detection method based on YOLOv5 and EWC according to claim 1, characterized in that: The specific steps of step S2 are: inputting a new data set, the new data set including new data that does not belong to the first data set, using the base model to predict pseudo labels of targets in the first data set that are related to the first data set, the pseudo labels including category information and location information of the targets; The pseudo labels are fused with the labels of the new data set to form the second data set.

5. The incremental target detection method based on YOLOv5 and EWC according to claim 1, characterized in that: The regularized loss function is as follows: Where loss(object) is the loss function of YOLOv5, λ is a hyperparameter, is the weight importance of parameter i in the tth task, θ i represents the actual value of the i-th parameter, represents the optimal parameters obtained after training for the tth task.

6. The incremental target detection method based on YOLOv5 and EWC according to any one of claims 1 to 5, characterized in that: The specific method of adjusting the parameters of the base model is: S311 and EWC use Gaussian posteriors for each model parameter and obtain the Hessian matrix through multivariate Gaussian distribution and Laplace approximation; S312, estimate the diagonal structure of the Hessian to calculate the diagonal structure of each parameter θ i To break it down: in Denotes the Hessian matrix H t The diagonal matrix of S313. Calculate the minimum loss function from the first task training to the t+1th task: Among them, λ is a hyperparameter used to balance the elasticity and stability of the model.

7. The incremental target detection method based on YOLOv5 and EWC according to claim 6, characterized in that: The specific method of step S311 is: Set the first data set D 1:t Trained parameters Using the second dataset D t+1 Calculate the optimal parameters of the new model Maximize the posterior information of all learned data sets: p(θ|D 1:t+1 )∝p(D t+1 |θ)p(θ|D 1:t ) Among them, p(D t+1 |θ) is the likelihood value of the second data set, p(θ|D 1:t ) is the posterior probability; The posterior probability in S311 and the optimal parameters trained by the first data set are calculated by multivariate Gaussian distribution. The optimal parameters are expressed as: in, = -logp(θ|D 1:t )exist The Hessian matrix at ; Use Laplace approximation to calculate the Hessian matrix and the posterior probability p(θ|D 1:t+1 ): Combined, we get the recursive expression of the Hessian matrix:

8. The incremental target detection method based on YOLOv5 and EWC according to claim 6, characterized in that: The Hessian diagonal matrix can be calculated by the following recursive equation:

9. The incremental target detection method based on YOLOv5 and EWC according to any one of claims 1 to 5, characterized in that: The parameter weights are updated recursively using the formula: in represents the loss of the jth sample on the tth task, Represents the importance of all parameters learned until task t+1; for the first task, initialize the importance of parameters, let 10. An incremental target detection system based on YOLOv5 and EWC, characterized in that: The system comprises: A base model module, used to initialize the YOLOv5 network model using the first data set to obtain a base model; A new data acquisition module, used to use the base model to acquire pseudo labels in a new data set and update the new data set to obtain a second data set; A first model module, used for inputting the second data set into the base model with the EWC algorithm added, training the second data set using a regularized loss function and adjusting the parameters of the base model to obtain a first model; The incremental target detection network model module is used to update the parameter weights of the first model, record the first model as the base model, repeat steps S2 to S3 for training until the first model with adjusted parameters meets the training end conditions, and obtain a network model for incremental target detection.

Images