Incremental learning method, device and computer equipment based on target detection

Abstract

The present application relates to an incremental learning method, device and computer equipment based on target detection, wherein the method includes: determining a first model and a second model; the first model is a fully trained target detection model; the second model is obtained by freezing the weight parameters of the fully trained target detection model according to a preset weight ratio; the network structure of the first model and the second model is the same; using scene images as training samples, the first model in the first mode and the second model in the second mode are self-distilled to obtain a second target model. Through this application, the problem that the target detection model in the related art will gradually forget the pre-training knowledge during the learning process, resulting in a decrease in the adaptability to new scenes, is solved, and it is ensured that the forgetting of pre-training knowledge is reduced during the learning process, thereby improving the adaptability to new scenes.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an incremental learning method, apparatus and computer device based on target detection. Background Art

[0002] Traditional target detection methods usually require a large amount of labeled data to train the model, but in practical applications, it is difficult to obtain a sufficient amount of new labeled data due to the emergence of new targets or changes in system requirements. Therefore, an incremental learning method based on small sample learning is proposed to cope with this demand more efficiently. This incremental learning method can be applied to various target detection scenarios that require real-time updates and iterations, such as video surveillance, unmanned driving, and intelligent security. In these scenarios, new targets may gradually appear over time, or existing targets need to be detected and classified more accurately.

[0003] Currently, when a fully trained target detection model is applied to a new scene, there is a problem of poor generalization ability. At this time, it is necessary to collect new scene data as a training set to continuously update the target detection model. However, during the update, the target detection model will gradually forget the pre-training knowledge during the learning process, resulting in a decrease in the adaptability to the new scene.

[0004] There is a problem in related technologies that target detection models will gradually forget the pre-training knowledge during the learning process, resulting in a decrease in the adaptability to new scenarios. No effective solution has been proposed so far. Summary of the invention

[0005] In this embodiment, a method, apparatus and computer device for incremental learning based on target detection are provided to solve the problem in related technologies that the target detection model gradually forgets the pre-training knowledge during the learning process, resulting in a decrease in the adaptability to new scenarios.

[0006] In a first aspect, in this embodiment, a method for incremental learning based on target detection is provided, including:

[0007] Determine a first model and a second model; the first model is a fully trained target detection model; the second model is obtained by freezing the weight parameters of the fully trained target detection model according to a preset weight ratio; the first model and the second model have the same network structure;

[0008] Using scene images as training samples, self-distillation learning is performed on a first model in a first mode and a second model in a second mode to obtain a second target model; the first mode is that all weight parameters of the first model are fixed and not updated; the second mode is that the weight parameters in the second model are frozen and not updated, and the unfrozen weight parameters participate in self-distillation learning; the scene image includes an original scene image and a target scene image; the second target model is the second model that is finally trained.

[0009] In some embodiments, a scene image is used as a training sample, and self-distillation learning is performed on a first model in a first mode and a second model in a second mode to obtain a second target model, including:

[0010] Using the scene image as a training sample, inputting it into the first model in the first mode, and obtaining a first inference result;

[0011] Input the scene image as a training sample into the second model in the second mode to obtain a second inference result;

[0012] Constructing a back propagation global loss function of the second model according to the first inference result and the second inference result;

[0013] Based on the back-propagation global loss function, the second model is trained to obtain a second target model.

[0014] In some embodiments, constructing a back propagation global loss function of the second model according to the first reasoning result and the second reasoning result includes:

[0015] Determine the node similarity between the full network nodes of the first model and the second model according to the first node output feature in the first reasoning result and the second node output feature in the second reasoning result; the first node output feature is the node output feature of the full network nodes in the second model; the second node output feature is the node output feature of the full network nodes in the second model;

[0016] Determine an output loss function of a difference between prediction results of the first model and prediction results of the second model according to a first prediction result in the first reasoning result and a second prediction result in the second reasoning result;

[0017] Constructing a back propagation global loss function of the second model according to the node similarity and the output loss function;

[0018] Based on the back-propagation global loss function, loss function convergence training is performed on the second model to obtain a second target model.

[0019] In some embodiments, constructing a back propagation global loss function of the second model according to the node similarity and the output loss function includes:

[0020] Construct a preliminary loss function based on the preset target detection function and target detection loss function;

[0021] Constructing a later loss function according to a preset parameter function, the node similarity and the output loss function;

[0022] According to the early loss function and the late loss function, a back-propagation global loss function of the second model is constructed.

[0023] In some of the embodiments, the target scene images account for 0 to 30% of the number of scene images.

[0024] In some embodiments, the method further comprises:

[0025] After the second model completes a round of training, the frozen weight ratio in the second model is adjusted.

[0026] In some embodiments, adjusting the frozen weight ratio in the second model includes:

[0027] According to a preset ratio threshold, the weight ratio frozen in the second model is adjusted upward or downward.

[0028] In some embodiments, the first model and the second model are one of a convolutional neural network model, a fully connected neural network model, and a generative adversarial network model.

[0029] In some embodiments, the method further comprises:

[0030] The image to be recognized of the target scene is input into the second target model for target detection to obtain a target detection result.

[0031] In a second aspect, an incremental learning device based on target detection is provided in this embodiment, comprising: a processing module and a training module;

[0032] The processing module is used to determine a first model and a second model; the first model is a fully trained target detection model; the second model is obtained by freezing the weight parameters of the fully trained target detection model according to a preset weight ratio; the first model and the second model have the same network structure;

[0033] The training module is used to use the scene image as a training sample, perform self-distillation learning on the first model in the first mode and the second model in the second mode, and obtain a second target model; the first mode is that all weight parameters of the first model are fixed and not updated; the second mode is that the weight parameters in the second model are frozen and not updated, and the unfrozen weight parameters participate in self-distillation learning; the scene image includes an original scene image and a target scene image; the second target model is the second model that is finally trained.

[0034] In a third aspect, a computer device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the incremental learning method based on target detection described in the first aspect is implemented.

[0035] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored, and when the program is executed by a processor, the incremental learning method based on target detection described in the first aspect above is implemented.

[0036] Compared with the related art, the incremental learning method, device and computer equipment based on target detection provided in this embodiment determine the first model and the second model; the first model is a fully trained target detection model; the second model is obtained by freezing the weight parameters of the fully trained target detection model according to a preset weight ratio; the network structure of the first model and the second model is the same; using the scene image as the training sample, the first model in the first mode and the second model in the second mode are self-distilled to obtain the second target model; the first mode is that all weight parameters of the first model are fixed and not updated; the second mode is that the frozen weight parameters in the second model are fixed and not updated, and the unfrozen weight parameters participate in the self-distillation learning; the scene image includes the original scene image and the target scene image; the second target model is the second model that is finally trained, which solves the problem in the related art that the target detection model will gradually forget the pre-training knowledge during the learning process, resulting in a decrease in the adaptability to the new scene. In the adaptation training of the new scene, the network structure of the target detection model should not be changed, and the first model is used to ensure that the forgetting of the pre-training knowledge is reduced during the learning process, thereby improving the adaptability to the new scene.

[0037] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0039] Figure 1 It is a hardware structure block diagram of a terminal device of an incremental learning method based on target detection provided by an embodiment of the present application;

[0040] Figure 2 is a flow chart of an incremental learning method based on target detection provided by an embodiment of the present application;

[0041] Figure 3 is a flowchart of step S220;

[0042] Figure 4 is a schematic diagram of a target detection function provided in an embodiment of the present application during a training process;

[0043] Figure 5 is a schematic diagram of a parameter function provided in an embodiment of the present application during a training process;

[0044] Figure 6 It is a structural block diagram of an incremental learning device based on target detection provided in one embodiment of the present application.

[0045] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 210, processing module; 220, training module. DETAILED DESCRIPTION

[0046] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0047] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0048] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 is a hardware structure block diagram of a terminal of the incremental learning method based on target detection in this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0049] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the incremental learning method based on target detection in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0050] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0051] In this embodiment, an incremental learning method based on target detection is provided. Figure 2 is a flow chart of the incremental learning method based on target detection in this embodiment, such as Figure 2 As shown, the process includes the following steps:

[0052] Step S210, determining a first model and a second model; the first model is a fully trained target detection model; the second model is obtained by freezing the weight parameters of the fully trained target detection model according to a preset weight ratio; the first model and the second model have the same network structure;

[0053] Step S220, using the scene image as a training sample, performing self-distillation learning on the first model in the first mode and the second model in the second mode to obtain a second target model; the first mode is that all weight parameters of the first model are fixed and not updated; the second mode is that the frozen weight parameters in the second model are fixed and not updated, and the unfrozen weight parameters participate in self-distillation learning; the scene image includes the original scene image and the target scene image; the second target model is the second model that has been finally trained.

[0054] Specifically, the first model and the second model use the same neural network architecture, and the first model and the second model can be one of a convolutional neural network model, a fully connected neural network model, and a generative adversarial network model. For example, if the first model is a convolutional neural network model, then the second model is also a convolutional neural network model, and the network structures of the two convolutional neural network models are also the same.

[0055] Among them, the first model is a fully trained target detection model based on the original scene image as a training sample, and it is in the first mode as a teacher model; that is, during the execution of the method, all weight parameters of the first model are fixed and not updated, and will not change with the execution of the method implementation example, so as to ensure that the pre-trained knowledge can be retained from the source when adapting to the new scene. The second model is a copy of the first model, and the weight parameters of the fully trained target detection model will be frozen according to the preset weight ratio; it is in the second mode as a student model, that is, during the execution of the method, the frozen weight parameters in the first model are fixed and not updated, and the unfrozen weight parameters participate in self-distillation learning, and will continue to change with each round of training. Among them, the target detection model is pre-trained, which is based on a certain amount of labeled images and combined with the target detection loss function to complete the pre-training. The specific training process can be adjusted according to the specific network form or structure of the target detection model, and there is no restriction on this.

[0056] Among them, self-distillation means finding a teacher model without adding a large model, which can also provide effective gain information to the student model. The gain information provided is effective incremental information for the student model to improve the efficiency of the student model. Among them, the student model is applicable to the target scene. The teacher model is applicable to the original scene. Since the training samples of the teacher model only have the original scene images, the detection effect of the teacher model on the original scene is better than that of the student model. The source of the above gain information is the part where the teacher model detects better than the student model on the original scene.

[0057] When adapting to a new scene, the target scene image is obtained and added to the original scene image to form a scene image as a training sample. The training samples are respectively input into the first model in the first mode and the second model in the second mode. The first model is used as a teacher module to teach the second model as a student model, thereby completing self-distillation learning and obtaining a second target model; the second target model is the second model that is finally trained; thereby achieving rapid completion of the adaptation training of the model in the new scene without introducing more additional learning modules and without modifying the network structure, and is suitable for scene-based small sample incremental learning tasks that are prone to overfitting. The scene image includes the original scene image and the target scene image. It can be considered that: when adapting to a new scene, if the target scene image corresponding to the new scene is insufficient, sufficient training samples can be provided with the help of the original scene image corresponding to the first model.

[0058] In the related art, when a well-trained target detection model is applied to a new scene, there is a problem of poor generalization ability. At this time, it is necessary to collect new scene data as a training set to continuously update the target detection model. However, during the update, the target detection model will gradually forget the pre-training knowledge during the learning process, resulting in a decrease in the adaptability to the new scene. In this embodiment, the first model and the second model are first determined; the first model is a fully trained target detection model; the second model is obtained by freezing the weight parameters of the fully trained target detection model according to a preset weight ratio; the network structure of the first model and the second model is the same; then the scene image is used as a training sample, and the first model in the first mode and the second model in the second mode are self-distilled to obtain a second target model; the first mode is that all weight parameters of the first model are fixed and not updated; the second mode is that the frozen weight parameters in the second model are fixed and not updated, and the unfrozen weight parameters participate in self-distillation learning; the scene image includes the original scene image and the target scene image; the second target model is the second model that is finally trained, which solves the problem that the target detection model in the related art will gradually forget the pre-training knowledge during the learning process, resulting in a decrease in the adaptation ability of the new scene. In the adaptation training of the new scene, the first model is retained as the teacher model, and the pre-training knowledge can be retained from the source when adapting to the new scene; and the first model is used as a teacher module to teach the second model as a student model, thereby completing the self-distillation learning to obtain the second target model. The frozen weight parameters in the second model are continuously updated to ensure that the pre-training knowledge is forgotten less during the learning process, thereby improving the adaptability to new scenarios.

[0059] The following is a detailed description of the above steps:

[0060] In some of the embodiments, Figure 3As shown, in step S220, the scene image is used as a training sample, and self-distillation learning is performed on the first model in the first mode and the second model in the second mode to obtain a second target model, including the following steps:

[0061] Step S221, taking the scene image as a training sample, inputting it into a first model in a first mode, and obtaining a first inference result;

[0062] Step S222, using the scene image as a training sample, inputting it into a second model in a second mode to obtain a second inference result;

[0063] Step S223, constructing a back propagation global loss function of the second model according to the first inference result and the second inference result;

[0064] Step S224: training the second model based on the back-propagation global loss function to obtain a second target model.

[0065] Specifically, the training samples are respectively input into a first model in a first mode and a second model in a second mode, and the first model outputs a first inference result; and the second model outputs a second inference result.

[0066] Among them, the first reasoning result and the second reasoning result both include the node output features of the full network nodes in their respective models; since the network structures of the first model and the second model are the same, the node output features in the two reasoning results are strongly correlated; for example: there are three full network nodes in the network structure of the first model and the second model (removing the input node and the output node, and the remaining intermediate nodes), then both reasoning results also have the node output features corresponding to these three nodes. The first reasoning result and the second reasoning result also include other data such as the prediction results output by their respective models.

[0067] The back propagation global loss function of the second model is constructed according to the first reasoning result and the second reasoning result, which can make full use of the first reasoning result of the teacher model to guide the second reasoning result of the student model; and with the iteration of the training cycle, the back propagation global loss function will be dynamically adjusted synchronously. Due to the introduction of constraints in the reasoning results, overfitting can be effectively avoided, and the forgetting of old knowledge in the training of the second model can be completed to obtain the second target model.

[0068] Through this embodiment, the constraints introduced in the reasoning results can effectively avoid the occurrence of overfitting and complete the forgetting of old knowledge by the second model.

[0069] In some embodiments, step S223, constructing a back propagation global loss function of the second model according to the first inference result and the second inference result, includes the following steps:

[0070] Determine the node similarity between the full network nodes of the first model and the second model according to the first node output feature in the first reasoning result and the second node output feature in the second reasoning result; the first node output feature is the node output feature of the full network node in the second model; the second node output feature is the node output feature of the full network node in the second model;

[0071] Determine an output loss function for a difference between prediction results of the first model and the second model according to a first prediction result in the first inference result and a second prediction result in the second inference result;

[0072] According to the node similarity and the output loss function, a back propagation global loss function of the second model is constructed;

[0073] Based on the back-propagation global loss function, the second model is trained to converge the loss function to obtain the second target model.

[0074] Specifically, the network structure of the first model and the second model is a YOLO V3 network structure as an example; the YOLO V3 network structure includes an input module (input), a feature extraction module (backbone), a feature fusion module (neck) and a prediction module (prediction). The first node output feature T-feature is the output of the feature extraction module and the feature fusion module in the first model; the first prediction result T-out is the output of the prediction module in the first model. Correspondingly, the second node output feature S-feature is the output of the feature extraction module and the feature fusion module in the second model; the second prediction result S-out is the output of the prediction module in the second model.

[0075] Among them, KL divergence, JS divergence and other calculation methods can be used to determine the node similarity between the full network nodes of the first model and the second model based on the first node output feature in the first reasoning result and the second node output feature in the second reasoning result, so as to align the output features of the full network nodes of the two models.

[0076] For example: The formula for calculating KL divergence is as follows:

[0077] KL(T-feature||S-feature)=Σ(T-feature(i)×log(T-feature(i) / S-feature(i)));

[0078] Where KL(T-feature||S-feature) represents the node similarity between the first model and the full network nodes of the first model; T-feature(i) represents the output of the i-th node in the first model; S-feature(i) represents the output of the i-th node in the second model.

[0079] Examples using other calculation methods are not repeated here.

[0080] Among them, calculation methods such as cross entropy and mean square error (MSE) can be used to implement an output loss function that determines the difference between the prediction results of the first model and the second model based on the first prediction result in the first reasoning result and the second prediction result in the second reasoning result, so as to distill the output ends of the two models.

[0081] For example: The formula for calculating cross entropy is as follows:

[0082] P = argmax(T-out);

[0083] Q = argmax(S-Out);

[0084] CrossEntropyLoss(P,Q)=-Σ(Q(i)×log(P(i)));

[0085] Wherein, CrossEntropyLoss(P, Q) represents the output loss function that uses cross entropy to calculate the difference between the output results of the first model and the second model; P is argmax(T-out), which represents the category with the highest probability in the output results of the first model; Q is argmax(S-Out), which represents the category with the highest probability in the output results of the second model; Σ represents the sum operation; Q(i) represents the probability of the category with index i in Q; P(i) represents the probability of the category with index i in P, and log represents the logarithm operation with base e.

[0086] Among them, the back-propagation global loss function of the second model is constructed by combining the node similarity and the output loss function, so that the back-propagation global loss function introduces constraints on the nodes of the entire network and the prediction results, which can effectively avoid the overfitting of the second model in each subsequent round of training, and complete the forgetting of old knowledge in the training of the second model to improve the training effect.

[0087] Through this embodiment, the node similarity and the output loss function are first determined, and then the back-propagation global loss function of the second model is constructed based on the node similarity and the output loss function; finally, based on the back-propagation global loss function, the second model is trained for loss function convergence to obtain the second target model, which can effectively avoid the overfitting of the second model in each subsequent round of training, and complete the forgetting of old knowledge in the training of the second model to improve the training effect.

[0088] In some embodiments, constructing a back propagation global loss function of the second model according to the node similarity and the output loss function comprises the following steps:

[0089] Construct a preliminary loss function based on the preset target detection function and target detection loss function;

[0090] Construct the post-loss function based on the preset parameter function, node similarity and output loss function;

[0091] According to the early loss function and the late loss function, the back propagation global loss function of the second model is constructed.

[0092] Among them, the preset target detection function is expressed as W_objloss; the target detection loss function is expressed as Loss_mx, and it is the loss function set by the target detection model during the training process, then the early loss function can be expressed as: W_objloss×(Loss_mx);

[0093] Among them, the preset parameter function is expressed as W_kd, then the later loss function constructed according to the preset parameter function, node similarity and output loss function can be expressed as W_kd×(CrossEntropyLoss(T-out,S-out)+KL(T-feature||S-feature)).

[0094] After obtaining the early loss function and the late loss function, the back propagation global loss function of the second model can be expressed as:

[0095]

[0096] Among them, the expression of the target detection function W_objloss is: W_objloss=-sigmoid(epoch); sigmoid (x) is a function, which is expressed as sigmoid(x)=1 / (1+exp(-x)); epoch represents the number of training rounds, completing a forward calculation and back propagation process. The change curve of the target detection function W_objloss is as follows Figure 4As shown in the figure, the horizontal axis is the number of training rounds, and the vertical axis is the size of the target detection function; as the number of training rounds increases, the target detection function changes from large to small. In other words, in the early stage of training, when the number of training rounds is relatively small, the target detection function is relatively large, and the target detection loss function accounts for a relatively large proportion of the back-propagation global loss function. At this time, the weight parameters involved in the training in the second model are updated according to the back-propagation global loss function, and the focus is on the learning of the second model corresponding to the target detection loss function on the small data task in the new scene.

[0097] The expression of parameter function W_kd is: W_kd=max(0.5,sigmoid(epoch)); the change curve of parameter function W_kd is as follows Figure 5 As shown in the figure, the horizontal axis is the number of training rounds, and the vertical axis is the size of the parameter function; as the number of training epochs increases, the parameter function W_kd increases. That is to say, in the middle and late stages of training, when the number of training rounds is relatively large, the parameter function becomes relatively large. At this time, the late loss function accounts for the majority of the back propagation global loss function. Optimizing this part of the loss mainly makes the output of the second model similar to the output of the first model, so that the second model learns old knowledge from the first model, thereby reducing the second model's forgetting of old knowledge.

[0098] In this embodiment, during the accumulation of the loss function, different weights are set for the target detection function, parameter function, and node similarity, where the weight of node similarity can be 1; then as the number of training rounds is added, the corresponding weights will also change synchronously, thereby dynamically adjusting the back propagation global loss function and improving efficiency. In other words, this embodiment can dynamically configure different loss function ratios at different stages according to the particularity of the fitting task during the training process to correct the degree of forgetting of historical knowledge.

[0099] Through this embodiment, in the early stage of training the second model, the model training converges faster in order to quickly complete the early model learning; therefore, at this stage, with the help of the early loss function in the back-propagation global loss function, the training task focuses on the second model's own capabilities, and there is no need to pay too much attention to the output provided by the first model and the corresponding feature alignment loss; in the middle and late stages of training, with the help of the late loss function in the back-propagation global loss function, as the number of iterations increases, the model task gradually becomes prone to overfitting. At this time, the first model is used to evaluate the global features of the second model, thereby preventing knowledge from being forgotten.

[0100] In some of the embodiments, the target scene images account for 0 to 30% of the scene images.

[0101] Specifically, the target scene image accounts for more than 0 in the number of scene images, or the target scene image accounts for less than or equal to 30% in the number of scene images. It can be considered that even when the number of target scene images is extremely small, the training of the second model can be completed to obtain the second target model. The larger the proportion of the target scene image in the number of scene images, the better the training effect can be considered, and there is no restriction on the specific proportion.

[0102] Through this embodiment, the training of the second model is achieved with a small number of target scene images.

[0103] In some of the embodiments, the incremental learning method based on target detection further includes the following steps:

[0104] After the second model completes a round of training, the proportion of frozen weights in the second model is adjusted.

[0105] Specifically, the training of the second model is carried out in rounds. For example, if there are 300 images in the training sample, one round of training is considered to be completed after traversing the 300 images once. At this time, the frozen weight ratio in the second model can be adjusted. For example: if the weight ratio in the first round is 90%, then only 10% of the weight parameters in the first round will participate in the training; if the weight ratio in the second round is adjusted to 85%, then 85% of the weight parameters will be randomly frozen in the second round, and the unfrozen 15% of the weight parameters will participate in the training. For example: if the weight ratio in the first round is 80%, then 20% of the weight parameters in the first round will participate in the training; if the weight ratio in the second round is adjusted to 85%, then 85% of the weight parameters will be randomly frozen in the second round, and the unfrozen 15% of the weight parameters will participate in the training.

[0106] Through this embodiment, the weight ratio can be adjusted to further improve the training effect.

[0107] In some of the embodiments, adjusting the frozen weight ratio in the second model includes:

[0108] According to the preset ratio threshold, the frozen weight ratio in the second model is adjusted up or down.

[0109] Specifically, after the second model completes a round of training, the frozen weight ratio in the second model can be adjusted up or down according to the preset ratio threshold. Among them, the ratio threshold can be 5%-20%, etc., and there is no restriction on this. In other words, the frozen weight ratio in the second model can be adjusted up or down according to the ratio threshold in each round, and the ratio threshold of each round can also be adaptively adjusted. For example: the weight ratio in the first round is 90%, so only 10% of the weight parameters in the first round participate in the training; in the second round, the ratio threshold is 10%, and the weight ratio needs to be lowered; then the weight ratio in the second round is adjusted to 80%, and 80% of the weight parameters will be randomly frozen in the second round, and the unfrozen 20% of the weight parameters will participate in the training. The ratio threshold in the third round is 15%, and the weight ratio needs to be lowered; then the weight ratio in the second round is adjusted to 65%, and 65% of the weight parameters will be randomly frozen in the second round, and the unfrozen 35% of the weight parameters will participate in the training.

[0110] Through this embodiment, the weight ratio can be adjusted by the ratio threshold to correspondingly adjust the weight of the loss function to further improve the training effect.

[0111] In some of the embodiments, the incremental learning method based on target detection further includes the following steps:

[0112] The image to be recognized of the target scene is input into the second target model for target detection to obtain a target detection result.

[0113] Specifically, after completing the adaptation training of the second target model, the image to be identified of the target scene can be directly input into the second target model for target detection, and the target detection result can be obtained quickly and accurately. The target detection result can be the classification result of the target. For example, in the field of security, the result of whether there is a target in the image to be identified is output.

[0114] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0115] In this embodiment, an incremental learning device based on target detection is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated hereafter. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware of predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0116] Figure 6is a structural block diagram of the incremental learning device based on target detection in this embodiment, such as Figure 6 As shown, the device includes: a processing module 210 and a training module 220;

[0117] Processing module 210, used to determine a first model and a second model; the first model is a fully trained target detection model; the second model is obtained by freezing the weight parameters of the fully trained target detection model according to a preset weight ratio; the first model and the second model have the same network structure;

[0118] The training module 220 is used to use the scene image as a training sample, perform self-distillation learning on the first model in the first mode and the second model in the second mode, and obtain a second target model; the first mode is that all weight parameters of the first model are fixed and not updated; the second mode is that the frozen weight parameters in the second model are fixed and not updated, and the unfrozen weight parameters participate in the self-distillation learning; the scene image includes the original scene image and the target scene image; the second target model is the second model that has been finally trained.

[0119] Through the above-mentioned device, the problem in the related art that the target detection model will gradually forget the pre-training knowledge during the learning process, resulting in a decrease in the adaptability to new scenes, is solved. In the adaptation training of the new scene, the network structure of the target detection model is not changed. The first model is used to ensure that the forgetting of pre-training knowledge is reduced during the learning process, thereby improving the adaptability to the new scene.

[0120] In some of the embodiments, the training module 220 is further configured to input the scene image as a training sample into a first model in a first mode to obtain a first inference result;

[0121] Input the scene image as a training sample into a second model in a second mode to obtain a second inference result;

[0122] Constructing a back propagation global loss function of the second model according to the first inference result and the second inference result;

[0123] Based on the back-propagation global loss function, the second model is trained to obtain the second target model.

[0124] In some of the embodiments, the training module 220 is further used to determine the node similarity between the full network nodes of the first model and the second model according to the first node output feature in the first reasoning result and the second node output feature in the second reasoning result; the first node output feature is the node output feature of the full network node in the second model; the second node output feature is the node output feature of the full network node in the second model;

[0125] Determine an output loss function for a difference between prediction results of the first model and the second model according to a first prediction result in the first inference result and a second prediction result in the second inference result;

[0126] According to the node similarity and the output loss function, a back propagation global loss function of the second model is constructed;

[0127] Based on the back-propagation global loss function, the second model is trained to converge the loss function to obtain the second target model.

[0128] In some of the embodiments, the training module 220 is further used to construct a preliminary loss function according to a preset target detection function and a target detection loss function;

[0129] Construct the post-loss function based on the preset parameter function, node similarity and output loss function;

[0130] According to the early loss function and the late loss function, the back propagation global loss function of the second model is constructed.

[0131] In some of the embodiments, the target scene images account for 0 to 30% of the scene images.

[0132] In some of the embodiments, the target detection-based incremental learning device further includes: a reset module;

[0133] The reset module is used to adjust the weight ratio frozen in the second model after the second model completes a round of training.

[0134] In some of the embodiments, the reset module is further used to increase or decrease the weight ratio frozen in the second model according to a preset ratio threshold.

[0135] In some of the embodiments, the first model and the second model are one of a convolutional neural network model, a fully connected neural network model, and a generative adversarial network model.

[0136] In some of the embodiments, the target detection-based incremental learning device further includes: a target detection module;

[0137] The target detection module is used to input the image to be identified of the target scene into the second target model for target detection to obtain the target detection result.

[0138] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0139] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0140] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0141] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0142] S1, determining a first model and a second model; the first model is a fully trained target detection model; the second model is obtained by freezing the weight parameters of the fully trained target detection model according to a preset weight ratio; the first model and the second model have the same network structure;

[0143] S2, using the scene image as a training sample, performing self-distillation learning on the first model in the first mode and the second model in the second mode to obtain a second target model; the first mode is that all weight parameters of the first model are fixed and not updated; the second mode is that the frozen weight parameters in the second model are fixed and not updated, and the unfrozen weight parameters participate in self-distillation learning; the scene image includes the original scene image and the target scene image; the second target model is the second model that has been finally trained.

[0144] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0145] In addition, in combination with the incremental learning method based on target detection provided in the above embodiments, a storage medium may also be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any incremental learning method based on target detection in the above embodiments is implemented.

[0146] It should be understood that the specific embodiments described here are only used to explain this application, rather than to limit it. According to the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0147] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0148] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0149] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. An incremental learning method based on target detection, characterized in that: include: determining a first model and a second model; The first model is a fully trained target detection model; The second model is obtained by freezing the weight parameters of the fully trained target detection model according to a preset weight ratio; The network structure of the first model is the same as that of the second model; Using the scene image as a training sample, self-distillation learning is performed on the first model in the first mode and the second model in the second mode to obtain a second target model; the first mode is that all weight parameters of the first model are fixed and not updated; the second mode is that the weight parameters frozen in the second model are fixed and not updated, and the unfrozen weight parameters participate in self-distillation learning; the scene image includes an original scene image and a target scene image; the second target model is the second model that is finally trained; The scene image is used as a training sample, and the first model in the first mode and the second model in the second mode are self-distilled to obtain a second target model, including: Using the scene image as a training sample, inputting it into the first model in the first mode, and obtaining a first inference result; Input the scene image as a training sample into the second model in the second mode to obtain a second inference result; Based on the back-propagation global loss function, the second model is trained to obtain a second target model; Constructing a node similarity and output loss function according to the first reasoning result and the second reasoning result; the first reasoning result and the second reasoning result both include node output features output by all network nodes in their respective models; According to the first reasoning result and the second reasoning result, constructing a node similarity and outputting a loss function includes: Determine the node similarity between the full network nodes of the first model and the second model according to the first node output feature in the first reasoning result and the second node output feature in the second reasoning result; the first node output feature is the node output feature output by the full network nodes in the first model; the second node output feature is the node output feature output by the full network nodes in the second model; Determine an output loss function of a difference between prediction results of the first model and prediction results of the second model according to a first prediction result in the first reasoning result and a second prediction result in the second reasoning result; The second model is trained based on the back-propagation global loss function to obtain a second target model, including: Constructing a back propagation global loss function of the second model according to the early loss function and the late loss function; The preliminary loss function is constructed by a preset target detection function and a target detection loss function; The late loss function is composed of a preset parameter function, the node similarity and the output loss function; Based on the back-propagation global loss function, loss function convergence training is performed on the second model to obtain a second target model.

2. The incremental learning method based on target detection according to claim 1, characterized in that: The target scene images account for 0 to 30% of the number of scene images.

3. The incremental learning method based on target detection according to claim 1, characterized in that: The method further comprises: After the second model completes a round of training, the frozen weight ratio in the second model is adjusted.

4. The incremental learning method based on target detection according to claim 3 is characterized in that: The frozen weight ratio in the second model is adjusted, including: According to a preset ratio threshold, the weight ratio frozen in the second model is adjusted upward or downward.

5. The incremental learning method based on target detection according to claim 1, characterized in that: The first model and the second model are one of a convolutional neural network model, a fully connected neural network model and a generative adversarial network model.

6. The incremental learning method based on target detection according to any one of claims 1 to 5, characterized in that: The method further comprises: The image to be recognized of the target scene is input into the second target model for target detection to obtain a target detection result.

7. An incremental learning device based on target detection, characterized in that: include: Processing module and training module; The processing module is used to determine the first model and the second model; The first model is a fully trained target detection model; The second model is obtained by freezing the weight parameters of the fully trained target detection model according to a preset weight ratio; The network structure of the first model is the same as that of the second model; The training module is used to use the scene image as a training sample, perform self-distillation learning on the first model in the first mode and the second model in the second mode, and obtain a second target model; the first mode is that all weight parameters of the first model are fixed and not updated; the second mode is that the weight parameters in the second model are frozen and not updated, and the unfrozen weight parameters participate in self-distillation learning; the scene image includes an original scene image and a target scene image; the second target model is the second model that is finally trained; The training module is further configured to input the scene image as a training sample into the first model in the first mode to obtain a first inference result; Input the scene image as a training sample into the second model in the second mode to obtain a second inference result; Based on the back-propagation global loss function, the second model is trained to obtain a second target model; Constructing a node similarity and output loss function according to the first reasoning result and the second reasoning result; the first reasoning result and the second reasoning result both include node output features output by all network nodes in their respective models; The training module is further used to determine the node similarity between the first model and the full network node of the first model according to the first node output feature in the first reasoning result and the second node output feature in the second reasoning result; the first node output feature is the node output feature output by the full network node in the first model; The second node output feature is a node output feature output by all network nodes in the second model; Determine an output loss function of a difference between prediction results of the first model and prediction results of the second model according to a first prediction result in the first reasoning result and a second prediction result in the second reasoning result; The training module is further used to construct a back propagation global loss function of the second model according to the early loss function and the late loss function; The preliminary loss function is constructed by a preset target detection function and a target detection loss function; The late loss function is composed of a preset parameter function, the node similarity and the output loss function; Based on the back-propagation global loss function, loss function convergence training is performed on the second model to obtain a second target model.

8. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps of the incremental learning method based on target detection according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the incremental learning method based on target detection described in any one of claims 1 to 6 are implemented.

Images