Continuous target detection method and system based on class feature and common feature decoupling

Through the decoupling method of category features and common features, combined with knowledge distillation and inverting cross-entropy loss, the problem of insufficient discrimination ability of new and old categories in the existing technology is solved, improving the accuracy of continuous object detection and reducing model retraining frequency.

CN118941774APending Publication Date: 2024-11-12NANJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411010785.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing continuous object detection method lacks the ability to discriminate similar features of old and new categories, resulting in poor performance and cannot guarantee that stored instances provide diverse features, resulting in frequent retraining of the model when facing new data, resulting in wasting time and material resources. .

Method used

Through a decoupling method based on category features and common features, the old category instances are screened using information entropy and confidence thresholds, and the category features and common feature regions are divided, and the cross-entropy loss is combined with knowledge distillation and inverted cross-entropy loss, and the model parameters are updated to improve the new and old categories. Discriminant ability and trait diversity.

Benefits of technology

Improves the accuracy of continuous object detection, reduces the need for retraining models for new data streams, and reduces time and material overhead.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941774A_ABST
    Figure CN118941774A_ABST
Patent Text Reader

Abstract

The invention discloses a continuous target detection method and system based on class feature and public feature decoupling, and relates to the technical field of continuous target detection, and the method comprises the steps: inputting an old class instance into a pre-established old model, outputting the confidence degree of each region, calculating the information entropy according to the confidence degree of each region, and carrying out the calculation of the information entropy; selecting small information entropy old class instances and inputting the instances into a memory; the storage selects stored old-class instances similar to the new-class instances in appearance, and the old-class instances are pasted into the pictures containing the new-class instances; dividing all regions into regions containing category features and regions containing public features on the basis of confidence coefficients of a pre-established old model and a new model in iterative updating on all regions of old-class instances, and calculating confidence coefficient consistency loss and inverse cross entropy loss of the new and old models on the category feature regions; and updating by using the total loss of the model to obtain an updated new model, and carrying out continuous target detection by using the updated new model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of persistent target detection, and in particular to a persistent target detection method and system based on decoupling of category features and common features. Background Art

[0002] With the rapid development of convolutional neural networks, the accuracy of target detection tasks has been greatly improved. However, most current target detection methods are based on static batch big data for training, while in actual applications, the data that various industries need to process are mostly dynamic, open, and unevenly distributed data streams. The model obtained by this batch learning training can only recognize categories that have appeared in the training set. For categories that do not appear in the training set, the model can only be retrained using a new training set containing new categories, which not only wastes a lot of time and material resources, but also causes catastrophic forgetting, that is, the performance of the model on the old categories previously learned is significantly reduced. In order to solve this problem, researchers began to introduce the continuous learning paradigm into the target detection task, trying to enable the model to continuously use the newly arrived data to learn new information, learn to handle new tasks, and maintain the ability to handle old tasks.

[0003] Recently, many continuous object detection methods have tried to use knowledge distillation technology to constrain the confidence of new and old models on stored old class instances. However, these methods have two shortcomings: first, the existing distillation methods lack the ability to distinguish similar features between new and old categories, resulting in poor performance of new models on similar categories; second, the existing methods select old class instances for storage based on instance-level features, which cannot guarantee that the stored instances can provide diverse features for knowledge distillation. Therefore, the key to the continuous object detection task lies in how to strengthen the ability to distinguish similar new and old categories, and how to select old class instances with diverse features. Summary of the invention

[0004] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a method and system for continuous target detection based on decoupling of category features and common features.

[0005] In a first aspect, the purpose of the present invention can be achieved by the following technical solution: a method for detecting a persistent target based on decoupling of category features and common features, the method comprising the following steps:

[0006] Receive old category instances, input the old category instances into the pre-established old model, output the confidence level on each region, calculate the information entropy according to the confidence level on each region, set the information entropy threshold, and input the old category instances that are less than the information entropy threshold into the memory;

[0007] receiving a picture containing a new category instance, the memory selects a stored old category instance similar in appearance to the new category instance, and pastes the old category instance into the picture;

[0008] Based on the confidence of the pre-established old model and the new model in iterative updating on each region of the old class instance, all regions are divided into regions containing category features and regions containing common features;

[0009] Based on the regions containing category features and the regions containing common features, knowledge distillation is used to generate the confidence consistency loss of the new and old models on the category feature regions. Based on the regions containing common features, the inverse cross entropy loss is generated based on the pre-established inverse cross entropy constraint model.

[0010] The confidence consistency loss and the inverse cross entropy loss are used to calculate the total model loss, and the new model parameters are updated using the total model loss to obtain the updated new model, which is then used for continuous target detection.

[0011] In combination with the first aspect, in some implementations of the first aspect, the method further includes: a process of selecting and storing the old category instance is as follows:

[0012] For each old class instance, it is cut into nine regions, each region is fed into the old model, nine category confidences are output, and the information entropy of these nine category confidences is calculated;

[0013] After obtaining the information entropy of all old class instances, the instances are sorted from small to large according to their corresponding information entropy, an information entropy threshold is set, and old class instances corresponding to those less than the information entropy threshold are selected and sent to the memory.

[0014] In combination with the first aspect, in some implementations of the first aspect, the method further includes: a process in which the memory selects a stored old category instance that is similar in appearance to the new category instance:

[0015] Based on the input of the new class instance, it is input into the old model for reasoning. If the confidence of a certain old category output by the old model is greater than the set threshold, the new class instance is considered to be similar in appearance to the old class;

[0016] Based on each old class instance stored in the memory, old class instances similar in appearance to new class instances are extracted, and the old class instances are pasted next to the new class instances.

[0017] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the process of dividing all regions into regions containing category features and regions containing common features:

[0018] Based on the confidence of the old model in each area of ​​the old class instance, if the background confidence of a certain area is higher than the confidence of other classes, condition one is met; if the confidence of the old class in a certain area is greater than the set threshold, condition two is met;

[0019] Based on the confidence of the new model in each region of the old class instance, if the confidence of the new class in a certain region is greater than the confidence of the old class, then condition three is met;

[0020] The area that satisfies the above conditions 1, 2 and 3 at the same time is divided into an area containing common features, otherwise it is divided into an area containing category features.

[0021] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the confidence consistency loss and the inverse cross entropy loss are respectively:

[0022]

[0023] In the formula, N represents the number of category feature regions; and Represent the confidence of the old model and the new model on the category feature area respectively; represents the indicator function, which has a value of 1 when the condition in the square brackets is met and a value of 0 when it is not met; δ represents the set threshold;

[0024] The inverse cross entropy loss is:

[0025]

[0026] In the formula, M represents the number of common feature regions. Represents the confidence of the new model in a common feature area. Represents the old class confidence of the new model in a common feature area, Represents the background confidence of the new model in a common feature area.

[0027] In combination with the first aspect, in some implementations of the first aspect, the method further includes: a process of calculating the total loss of the model:

[0028] Based on the knowledge distillation technology, the confidence difference between the new and old models in each category feature area is calculated to obtain the confidence consistency loss L conf ;

[0029] Based on the inverted cross entropy technology, the background confidence, old class confidence and new class confidence of the new model in each common feature area are inverted to obtain the inverted confidence, and the cross entropy loss is calculated with the balanced label to obtain the inverted cross entropy loss L rce ;

[0030] Calculate the standard object detection loss Ldet , and the auxiliary unbiased distillation loss L unb ;

[0031] Calculate the total loss L of model training tota :

[0032] L total =L det +λ 1 L unb +λ 2 L conf +λ 3 L rce

[0033] Among them, λ 1 , 2 and λ 3 It is the balance coefficient used to balance various losses.

[0034] In a second aspect, in order to achieve the above-mentioned object, the present invention discloses a continuous target detection system based on decoupling of category features and common features, comprising:

[0035] An instance storage module is used to receive old category instances, input the old category instances into a pre-established old model, output the confidence level on each region, calculate the information entropy according to the confidence level on each region, set an information entropy threshold, and input the old category instances that are less than the information entropy threshold into a memory;

[0036] A new and old mixing module is used to receive a picture containing a new category instance, and the memory selects a stored old category instance that is similar in appearance to the new category instance, and pastes the old category instance into the picture;

[0037] A region division module is used to divide all regions into regions containing category features and regions containing common features based on the confidence of the pre-established old model and the new model on each region of the old class instance;

[0038] A loss calculation module is used to generate the confidence consistency loss of the new and old models on the category feature areas based on the areas containing category features and the areas containing common features by using knowledge distillation, and to generate the inverse cross entropy loss based on the areas containing common features based on the pre-established inverse cross entropy constraint model;

[0039] The new model updating module is used to calculate the total model loss using the confidence consistency loss and the inverse cross entropy loss, update the new model parameters using the total model loss, obtain the updated new model, and use the updated new model for continuous target detection.

[0040] In conjunction with the second aspect, in some implementations of the second aspect, the system further includes: a process of selecting and storing old category instances in the instance storage module is as follows:

[0041] For each old class instance, it is cut into nine regions, each region is fed into the old model, nine category confidences are output, and the information entropy of these nine category confidences is calculated;

[0042] After obtaining the information entropy of all old class instances, sort the instances according to their corresponding information entropy from small to large, set the information entropy threshold, and select the old class instances corresponding to the information entropy threshold that is smaller than the information entropy threshold and send them to the memory;

[0043] The process of selecting the old category instance stored in the memory of the new and old mixed module that is similar in appearance to the new category instance:

[0044] Based on the input image containing the new class instance, it is input to the old model for reasoning. If the confidence of a certain old category output by the old model is greater than the set threshold, the new class instance is considered to be similar in appearance to the old class;

[0045] Based on each old class instance stored in the memory, extract the old class instance that is similar in appearance to the new class instance, and paste the old class instance next to the new class instance;

[0046] The process of dividing all regions into regions with category features and regions with common features in the region division module:

[0047] Based on the confidence of the old model in each area of ​​the old class instance, if the background confidence of a certain area is higher than the confidence of other classes, condition one is met; if the confidence of the old class in a certain area is greater than the set threshold, condition two is met;

[0048] Based on the confidence of the new model in each region of the old class instance, if the confidence of the new class in a certain region is greater than the confidence of the old class, then condition three is met;

[0049] The area that meets the above conditions 1, 2 and 3 at the same time is classified as an area with common features, otherwise it is classified as an area with category features;

[0050] The built-in belief consistency loss and inverse cross entropy loss in the loss calculation module are:

[0051]

[0052] In the formula, N represents the number of category feature regions; and Represent the confidence of the old model and the new model on the category feature area respectively; represents the indicator function, which has a value of 1 when the condition in the square brackets is met and a value of 0 when it is not met; δ represents the set threshold;

[0053] The inverse cross entropy loss is:

[0054]

[0055] In the formula, M represents the number of common feature regions. Represents the confidence of the new model in a common feature area. Represents the old class confidence of the new model in a common feature area, Represents the background confidence of the new model in a common feature area;

[0056] The process of calculating the total model loss in the new model update module:

[0057] Based on the knowledge distillation technology, the confidence difference between the new and old models in each category feature area is calculated to obtain the confidence consistency loss L conf ;

[0058] Based on the inverted cross entropy technology, the background confidence, old class confidence and new class confidence of the new model in each common feature area are inverted to obtain the inverted confidence, and the cross entropy loss is calculated with the balanced label to obtain the inverted cross entropy loss L rce ;

[0059] Calculate the standard object detection loss L det , and the auxiliary unbiased distillation loss L unb ;

[0060] Calculate the total loss L of model training tota :

[0061] L total =L det +λ 1 L unb +λ 2 L conf +λ 3 L rce

[0062] Among them, λ 1 , 2 and λ 3 It is the balance coefficient used to balance various losses.

[0063] In another aspect of the present invention, in order to achieve the above-mentioned purpose, a terminal device is disclosed, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the memory stores a computer program capable of running on the processor, and when the processor loads and executes the computer program, the continuous target detection method based on decoupling of category features and common features as described above is adopted.

[0064] In another aspect of the present invention, in order to achieve the above-mentioned purpose, a computer-readable storage medium is disclosed, in which a computer program is stored. When the computer program is loaded and executed by a processor, the continuous target detection method based on decoupling of category features and common features as described above is adopted.

[0065] Beneficial effects of the present invention:

[0066] The present invention can well solve the common problems in existing continuous target detection methods, namely, the lack of ability to distinguish similar new and old categories and the inability to provide old category instances containing diverse features, thereby improving the accuracy of continuous target detection tasks. It can also greatly reduce the material and time costs required to retrain the target detection model when facing a continuous incoming data stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0068] Figure 1 It is a schematic flow chart of the method of the present invention;

[0069] Figure 2 It is a structural block diagram of the present invention;

[0070] Figure 3 It is a schematic diagram of the system structure of the present invention;

[0071] Figure 4 It is a comparison chart of the verification effect of the present invention. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0073] Embodiment 1:

[0074] The following is an introduction to the relevant terms involved in the embodiments of the present application:

[0075] Knowledge distillation: Knowledge distillation (KD) is a classic model compression method. The core idea is to improve the performance of the student model without changing its structure by guiding the lightweight student model to "imitate" a teacher model (or an ensemble of multiple models) with better performance and more complex structure.

[0076] Information entropy represents the uncertainty of random variables. Simply put, the higher the data uncertainty (or the more random it is), the greater the information entropy of the corresponding data; the lower the data uncertainty (or the more certain it is), the smaller the information entropy of the corresponding data.

[0077] like Figure 1 As shown, the continuous target detection method based on the decoupling of category features and common features includes the following steps:

[0078] S101: receiving old category instances, inputting the old category instances into a pre-established old model, outputting the confidence level on each region, calculating information entropy according to the confidence level on each region, setting an information entropy threshold, and inputting old category instances that are less than the information entropy threshold into a memory;

[0079] For an instance of an old class c Its length and width are H and W respectively. Set the length and width to and The horizontal and vertical step lengths are and The sliding window of Divided into 9 areas and entered into the old model Get the confidence of 9 categories c;

[0080] Calculate the information entropy after normalizing the 9 confidence softmax Calculate the information entropy of each instance belonging to category c in turn, and sort them in order from small to large according to the information entropy, and select the first k instances to enter the memory. Follow the above operation for other old categories to get the final memory.

[0081] S102: receiving a picture containing a new category instance, selecting a stored old category instance similar in appearance to the new category instance from the memory, and pasting the old category instance into the picture;

[0082] For an input image containing an instance of a new class, input the new class instance into the old model In the above example, the confidence of the old model for each old class is obtained. If the confidence of an old class is greater than the set threshold, it is considered that the new class instance is similar in appearance to the old class.

[0083] Select an instance belonging to this old class in the memory, select an appropriate position and size and paste it into the input image to obtain a fused image that contains similar instances of both the old and new classes.

[0084] S103: based on the confidence of the pre-established old model and the new model on each region of the old class instance, all regions are divided into regions containing category features and regions containing common features;

[0085] Feed the fused image into the new model In the example, we can get several candidate regions on the instance of the old category c, and take a candidate region R i Input into the classification head of the new model to get the background confidence The sum of all old class confidences and the sum of all new class confidences Input all candidate regions into the classification head of the old model to get the background confidence Old category c confidence

[0086] when Greater than When the area R i Satisfy condition 1. When the area R i Satisfy condition 2. When the area R i Satisfy condition 3. When a candidate region satisfies the above three conditions at the same time, the region is classified as a region containing common features. Regions that do not meet the above three conditions at the same time are classified as regions containing category features.

[0087] Based on the regions containing category features and the regions containing common features, knowledge distillation is used to generate the confidence consistency loss of the new and old models on the category feature regions. Based on the regions containing common features, the inverse cross entropy loss is generated based on the pre-established inverse cross entropy constraint model.

[0088] Based on the knowledge distillation technology, the L2 distance of the confidence of the new and old models on the category feature area is calculated as the training loss, and the areas with smaller L2 distance are filtered out to construct the confidence consistency loss;

[0089] Confidence consistency loss L conf for:

[0090]

[0091] In the formula, N represents the number of category feature regions; and Represent the confidence of the old model and the new model on the category feature area respectively; represents the indicator function, which has a value of 1 when the condition in the square brackets is met and a value of 0 when it is not met; δ represents the set threshold.

[0092] Based on the inverted cross entropy technology, the background confidence, old class confidence and new class confidence of the new model in each common feature area are inverted to obtain the inverted category response, and the cross entropy loss is calculated with the balanced label (0, 0.5, 0.5) to obtain the inverted cross entropy loss L rce ;

[0093] Invert the cross entropy loss L rce for:

[0094]

[0095] In the formula, M represents the number of common feature regions. Represents the confidence of the new model in a common feature area. Represents the old class confidence of the new model in a common feature area, Represents the background confidence of the new model in a common feature area.

[0096] S105: Calculate the total model loss using the confidence consistency loss and the inverse cross entropy loss, use the total model loss to update the new model parameters, obtain an updated new model, and use the updated new model to perform continuous target detection.

[0097] Specifically, the present invention is further described below by way of embodiments:

[0098] Calculate the standard object detection loss L det , and the auxiliary unbiased distillation loss L commonly used in related work in this field unb ;

[0099] The standard object detection loss is L det :

[0100]

[0101] Among them, K represents the number of all candidate regions of the new class, C t Represents all new categories, Represents the category label of the candidate region, Represents the confidence of the category c output by the new model on the candidate region, l smoot_l1 represents the smooth L1 distance, Represents the coordinate regression value of category c output by the new model on the candidate area, Coordinate labels representing candidate regions.

[0102] The unbiased distillation loss L commonly used in related work in this fieldunb for:

[0103]

[0104] Among them, H represents the number of all old class candidate regions, |C t-1 | represents the number of all old classes, and Represent the background confidence of the old model and the new model respectively, and Represent the coordinate regression values ​​of the new model and the old model respectively.

[0105] The total loss L of model training tota The calculation formula is:

[0106] L total =L det +λ 1 L unb +λ 2 L conf +λ 3 L rce

[0107] Among them, λ 1 , 2 and λ 3 It is the balance coefficient used to balance various losses.

[0108] Embodiment 2: In the second aspect, as Figure 3 As shown, in order to achieve the above-mentioned purpose, the present invention discloses a continuous target detection system based on decoupling of category features and common features, including:

[0109] The instance storage module 11 is used to receive old category instances, input the old category instances into the pre-established old model, output the confidence level on each region, calculate the information entropy according to the confidence level on each region, set the information entropy threshold, and input the old category instances that are less than the information entropy threshold into the memory;

[0110] The new and old mixing module 12 is used to receive a picture containing a new category instance, select an old category instance stored in the memory that is similar in appearance to the new category instance, and paste the old category instance into the picture;

[0111] A region division module 13, for dividing all regions into regions containing category features and regions containing common features based on the confidence of the pre-established old model and the new model on each region of the old class instance;

[0112] The loss calculation module 14 is used to generate the confidence consistency loss of the new and old models on the category feature areas based on the areas containing category features and the areas containing common features by using knowledge distillation, and generate the inverse cross entropy loss based on the areas containing common features based on the pre-established inverse cross entropy constraint model;

[0113] The new model updating module 14 is used to calculate the total model loss using the confidence consistency loss and the inverse cross entropy loss, update the new model parameters using the total model loss, obtain the updated new model, and use the updated new model to perform continuous target detection.

[0114] In combination with the second aspect, in some implementations of the second aspect, the system further includes: a process of selecting and storing old category instances in the instance storage module 11 is as follows:

[0115] For each old class instance, it is cut into nine regions, each region is fed into the old model, nine category confidences are output, and the information entropy of these nine category confidences is calculated;

[0116] After obtaining the information entropy of all old class instances, sort the instances according to their corresponding information entropy from small to large, set the information entropy threshold, and select the old class instances corresponding to the information entropy threshold that is smaller than the information entropy threshold and send them to the memory;

[0117] The process of selecting the old category instance similar in appearance to the new category instance from the memory in the new-old mixing module 12:

[0118] Based on the new class instances in the input image, they are input to the old model for inference. If the confidence of a certain old category output by the old model is greater than the set threshold, the new class instance is considered to be similar in appearance to the old class.

[0119] Based on each old class instance stored in the memory, extract the old class instance that is similar in appearance to the new class instance, and paste the old class instance next to the new class instance;

[0120] The process of dividing all regions into regions containing category features and regions containing common features in the region division module 13:

[0121] Based on the confidence of the old model in each area of ​​the old class instance, if the background confidence of a certain area is higher than the confidence of other classes, condition one is met; if the confidence of the old class in a certain area is greater than the set threshold, condition two is met;

[0122] Based on the confidence of the new model in each region of the old class instance, if the confidence of the new class in a certain region is greater than the confidence of the old class, then condition three is met;

[0123] The area that meets the above conditions 1, 2 and 3 at the same time is classified as an area with common features, otherwise it is classified as an area with category features;

[0124] The built-in belief consistency loss and inverse cross entropy loss of loss calculation module 14 are:

[0125]

[0126] In the formula, N represents the number of category feature regions; and Represent the confidence of the old model and the new model on the category feature area respectively; represents the indicator function, which has a value of 1 when the condition in the square brackets is met and a value of 0 when it is not met; δ represents the set threshold;

[0127] The inverse cross entropy loss is:

[0128]

[0129] In the formula, M represents the number of common feature regions. Represents the confidence of the new model in a common feature area. Represents the old class confidence of the new model in a common feature area, Represents the background confidence of the new model in a common feature area;

[0130] The process of calculating the total model loss in the new model update module 15:

[0131] Based on the knowledge distillation technology, the confidence difference between the new and old models in each category feature area is calculated to obtain the confidence consistency loss L conf ;

[0132] Based on the inverted cross entropy technology, the background confidence, old class confidence and new class confidence of the new model in each common feature area are inverted to obtain the inverted confidence, and the cross entropy loss is calculated with the balanced label to obtain the inverted cross entropy loss L rce ;

[0133] Calculate the standard object detection loss L det , and the auxiliary unbiased distillation loss L commonly used in related work in this field unb ;

[0134] Calculate the total loss L of model training tota :

[0135] L total =L det +λ 1 L unb +λ 2 L conf +λ 3 L rce

[0136] Among them, λ 1 , 2 and λ 3 It is the balance coefficient used to balance various losses.

[0137] Embodiment 3: The third aspect, as Figure 4 As shown, in order to verify the effectiveness of the present invention, the visualization effects of the method of the present invention and the existing method are compared on verification picture 1 and verification picture 2 respectively.

[0138] In the first stage of training, both the existing method and the method of the present invention are trained on a number of pictures containing "bus" to obtain an established old model.

[0139] In the second stage of training, the old models of the existing method and the method of the present invention continue to be trained on several pictures containing the "train" to obtain a new model.

[0140] In the verification stage, the verification image 1 contains the old class instance "bus". The verification image 1 is input into the new model of the existing method to obtain a detection frame with a train confidence of 0.84 and a detection frame with a bus confidence of 0.95; the verification image 1 is input into the new model of the method of the present invention to obtain a detection frame with a bus confidence of 0.84 and a detection frame with a bus confidence of 0.95. The detection accuracy of the present invention for old class objects is verified.

[0141] In the verification stage, the second verification image contains a new class instance "train". The second verification image is input into the new model of the existing method, and a detection box with a confidence of 0.55 for a bus and a detection box with a confidence of 0.79 for a train is obtained; the second verification image is input into the new model of the method of the present invention, and a detection box with a confidence of 0.79 for a train and a detection box with a confidence of 0.98 for a train is obtained. The detection accuracy of the present invention for new class objects is verified.

[0142] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.

[0143] It needs to be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to execute the above method. The storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0144] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0145] The above shows and describes the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, the present disclosure may have various changes and improvements, and these changes and improvements fall within the scope of the present disclosure to be protected.

Claims

1. A continuous target detection method based on decoupling of category features and common features, characterized in that: The method comprises the following steps: Receive old category instances, input the old category instances into the pre-established old model, output the confidence level on each region, calculate the information entropy according to the confidence level on each region, set the information entropy threshold, and input the old category instances that are less than the information entropy threshold into the memory; receiving a picture containing a new category instance, the memory selects a stored old category instance similar in appearance to the new category instance, and pastes the old category instance into the picture; Based on the confidence of the pre-established old model and the new model in iterative updating on each region of the old class instance, all regions are divided into regions containing category features and regions containing common features; Based on the regions containing category features and the regions containing common features, knowledge distillation is used to generate the confidence consistency loss of the new and old models on the category feature regions. Based on the regions containing common features, the inverse cross entropy loss is generated based on the pre-established inverse cross entropy constraint model. The confidence consistency loss and the inverse cross entropy loss are used to calculate the total model loss, and the new model parameters are updated using the total model loss to obtain the updated new model, which is then used for continuous target detection.

2. The method for continuous target detection based on decoupling of category features and common features according to claim 1, characterized in that: The selection and storage process of the old category instance is as follows: For each old class instance, it is cut into nine regions, each region is fed into the old model, nine category confidences are output, and the information entropy of these nine category confidences is calculated; After obtaining the information entropy of all old class instances, the instances are sorted from small to large according to the corresponding information entropy, and the information entropy threshold is set, and the old class instances corresponding to the information entropy threshold that is smaller than the information entropy threshold are selected and sent to the memory.

3. The method for continuous target detection based on decoupling of category features and common features according to claim 1, characterized in that: The process of the memory selecting a stored old category instance that is similar in appearance to a new category instance: Based on the input of the new class instance, it is input into the old model for reasoning. If the confidence of a certain old category output by the old model is greater than the set threshold, the new class instance is considered to be similar in appearance to the old class; Based on each old class instance stored in the memory, old class instances similar in appearance to new class instances are extracted, and the old class instances are pasted next to the new class instances.

4. The method for continuous target detection based on decoupling of category features and common features according to claim 1, characterized in that: The process of dividing all regions into regions containing category features and regions containing common features: Based on the confidence of the old model in each area of ​​the old class instance, if the background confidence of a certain area is higher than the confidence of other classes, condition one is met; if the confidence of the old class in a certain area is greater than the set threshold, condition two is met; Based on the confidence of the new model in each region of the old class instance, if the confidence of the new class in a certain region is greater than the confidence of the old class, then condition three is met; The area that satisfies the above conditions 1, 2 and 3 at the same time is divided into an area containing common features, otherwise it is divided into an area containing category features.

5. The method for continuous target detection based on decoupling of category features and common features according to claim 1, characterized in that: The confidence consistency loss and the inverse cross entropy loss are: In the formula, N represents the number of category feature regions; and Represent the confidence of the old model and the new model on the category feature area respectively; represents the indicator function, which has a value of 1 when the condition in the square brackets is met and a value of 0 when it is not met; δ represents the set threshold; The inverse cross entropy loss is: Where M represents the number of common feature regions. Represents the confidence of the new model in a common feature area. Represents the old class confidence of the new model in a common feature area, Represents the background confidence of the new model in a common feature area.

6. The method for continuous target detection based on decoupling of category features and common features according to claim 1, characterized in that: The process of calculating the total loss of the model is as follows: Based on the knowledge distillation technology, the confidence difference between the new and old models in each category feature area is calculated to obtain the confidence consistency loss L conf ; Based on the inverted cross entropy technology, the background confidence, old class confidence and new class confidence of the new model in each common feature area are inverted to obtain the inverted confidence, and the cross entropy loss is calculated with the balanced label to obtain the inverted cross entropy loss L rce ; Calculate the standard object detection loss L det , and the auxiliary unbiased distillation loss L unb ; Calculate the total loss L of model training tota : L total =L det +λ1L unb +λ2L conf +λ3L rce Among them, λ1, λ2 and λ3 are balancing coefficients used to balance various losses.

7. A continuous target detection system based on decoupling of category features and common features, characterized in that: include: An instance storage module is used to receive old category instances, input the old category instances into a pre-established old model, output the confidence level on each region, calculate the information entropy according to the confidence level on each region, set an information entropy threshold, and input the old category instances that are less than the information entropy threshold into a memory; A new and old mixing module is used to receive a picture containing a new category instance, and the memory selects a stored old category instance that is similar in appearance to the new category instance, and pastes the old category instance into the picture; A region division module is used to divide all regions into regions containing category features and regions containing common features based on the pre-established old model and the confidence of the new model in the iterative update on each region of the old class instance; A loss calculation module is used to generate the confidence consistency loss of the new and old models on the category feature areas based on the areas containing category features and the areas containing common features by using knowledge distillation, and to generate the inverse cross entropy loss based on the areas containing common features based on the pre-established inverse cross entropy constraint model; The new model updating module is used to calculate the total model loss using the confidence consistency loss and the inverse cross entropy loss, update the new model parameters using the total model loss, obtain the updated new model, and use the updated new model for continuous target detection.

8. The continuous target detection system based on decoupling of category features and common features according to claim 7, characterized in that: The process of selecting and storing old category instances in the instance storage module is as follows: For each old class instance, it is cut into nine regions, each region is fed into the old model, nine category confidences are output, and the information entropy of these nine category confidences is calculated; After obtaining the information entropy of all old class instances, sort the instances according to their corresponding information entropy from small to large, set the information entropy threshold, and select the old class instances corresponding to the information entropy threshold that is smaller than the information entropy threshold and send them to the memory; The process of selecting the old category instance stored in the memory of the new and old mixed module that is similar in appearance to the new category instance: Based on the new class instances in the input image, the new class instances are input to the old model for inference. If the confidence of a certain old class output by the old model is greater than the set threshold, the new class instances are considered to be similar in appearance to the old class instances. Based on each old class instance stored in the memory, extract the old class instance that is similar in appearance to the new class instance, and paste the old class instance next to the new class instance; The process of dividing all regions into regions with category features and regions with common features in the region division module: Based on the confidence of the old model in each area of ​​the old class instance, if the background confidence of a certain area is higher than the confidence of other classes, condition one is met; if the confidence of the old class in a certain area is greater than the set threshold, condition two is met; Based on the confidence of the new model in each region of the old class instance, if the confidence of the new class in a certain region is greater than the confidence of the old class, then condition three is met; The area that meets the above conditions 1, 2 and 3 at the same time is classified as an area with common features, otherwise it is classified as an area with category features; The built-in belief consistency loss and inverse cross entropy loss in the loss calculation module are: In the formula, N represents the number of category feature regions; and Represent the confidence of the old model and the new model on the category feature area respectively; represents the indicator function, which has a value of 1 when the condition in the square brackets is met and a value of 0 when it is not met; δ represents the set threshold; The inverse cross entropy loss is: Where M represents the number of common feature regions. Represents the confidence of the new model in a common feature area. Represents the old class confidence of the new model in a common feature area, Represents the background confidence of the new model in a common feature area; The process of calculating the total model loss in the new model update module: Based on the knowledge distillation technology, the confidence difference between the new and old models in each category feature area is calculated to obtain the confidence consistency loss L conf ; Based on the inverted cross entropy technology, the background confidence, old class confidence and new class confidence of the new model in each common feature area are inverted to obtain the inverted confidence, and the cross entropy loss is calculated with the balanced label to obtain the inverted cross entropy loss L rce ; Calculate the standard object detection loss L det , and the auxiliary unbiased distillation loss L unb ; Calculate the total loss L of model training tota : L total =L det +λ1L unb +λ2L conf +λ3L rce Among them, λ1, λ2 and λ3 are balancing coefficients used to balance various losses.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: The memory stores a computer program that can be run on the processor. When the processor loads and executes the computer program, the continuous target detection method based on decoupling of category features and common features described in any one of claims 1 to 6 is adopted.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, characterized in that: When the computer program is loaded and executed by the processor, the continuous target detection method based on decoupling of category features and common features as described in any one of claims 1 to 6 is adopted.