Lightweight object detection method and device based on incremental learning

Abstract

The present invention provides a lightweight object detection method and device based on incremental learning. The lightweight object detection method based on incremental learning includes: obtaining an image to be tested and a target feature category image; inputting the image to be tested into a detection module to obtain multiple sub-feature images output by the detection module; inputting the multiple sub-feature images into a classification module to obtain feature expressions corresponding to the sub-feature images output by the classification module, as well as the category corresponding to the first target sub-feature image in the multiple sub-feature images; and determining the category corresponding to the second target sub-feature image based on the similarity between the feature expression corresponding to the second target sub-feature image in the multiple sub-feature images and the target feature category image. The lightweight object detection method based on incremental learning of the present invention can complete the recognition of new categories without repeated training, significantly improving the generalization ability and scalability of the model, and is suitable for scenarios involving constantly changing information flows.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a lightweight object detection method and device based on incremental learning. Background Art

[0002] When classifying and recognizing images, real-world scenarios such as security inspections, autonomous driving, and medical imaging often require models to be able to continuously process continuous information streams and respond to new categories. However, with traditional image recognition, the data categories used during training and testing are the same. A common solution is to continuously add new categories to the training dataset and retrain the model. However, this approach requires significant computational resources, new category data, and manual labeling. Summary of the Invention

[0003] The present invention provides a lightweight object detection method and device based on incremental learning, which is used to solve the defects of the existing technology in processing constantly changing continuous information streams, such as high computational cost and manual labeling cost, improve the generalization and scalability of the recognition model, and reduce the recognition cost.

[0004] The present invention provides a lightweight object detection method based on incremental learning, comprising:

[0005] Acquire an image to be tested and a target feature category image, wherein the target feature category image is an image corresponding to an untrained feature category in the image to be tested;

[0006] Inputting the image to be tested into a detection module, and obtaining a plurality of sub-feature images output by the detection module;

[0007] Inputting the plurality of sub-feature images into a classification module, obtaining feature expressions corresponding to the sub-feature images output by the classification module, and a category corresponding to a first target sub-feature image among the plurality of sub-feature images; the category corresponding to the first target sub-feature image is a trained feature category;

[0008] determining a category corresponding to a second target sub-feature image based on a similarity between a feature expression corresponding to a second target sub-feature image in the plurality of sub-feature images and the target feature category image; the category corresponding to the second target sub-feature image being an untrained feature category;

[0009] The output end of the detection module is connected to the input end of the classification module, the detection module is obtained after training based on sample images with sample sub-feature image labels, and the classification module is obtained after training based on sample sub-feature images with sample category labels.

[0010] According to a lightweight object detection method based on incremental learning provided by the present invention, determining the category corresponding to the second target sub-feature image based on the similarity between the feature expression corresponding to the second target sub-feature image in the multiple sub-feature images and the target feature category image includes:

[0011] Performing data enhancement processing on the target feature category image to generate an average mapping feature of N mapping features in a mapping feature space; wherein N is a positive integer;

[0012] generating a similarity between a feature expression corresponding to the second target sub-feature image and the average mapping feature;

[0013] When the similarity exceeds a target threshold, the category of the second target feature image is determined as the target feature category.

[0014] According to a lightweight object detection method based on incremental learning provided by the present invention, the detection module includes a first backbone network, the classification module includes a second backbone network, the first backbone network and the second backbone network have the same structure and share weight initialization.

[0015] According to a lightweight object detection method based on incremental learning provided by the present invention, the first backbone network and the second backbone network include GhostNet modules.

[0016] According to a lightweight object detection method based on incremental learning provided by the present invention, before acquiring the image to be detected and the target feature category image, the method includes:

[0017] Acquire multiple sample images and a sample sub-feature image label corresponding to each of the sample images;

[0018] Each of the sample images and the sample sub-feature image label corresponding to the sample image is taken as a first training sample to obtain a plurality of first training samples, and the detection module is trained using the plurality of first training samples.

[0019] According to a lightweight object detection method based on incremental learning provided by the present invention, before acquiring the image to be detected and the target feature category image, the method includes:

[0020] Obtaining a plurality of sample sub-feature images and a category label corresponding to each of the sample sub-feature images;

[0021] Each of the sample sub-feature images and the category label corresponding to the sample sub-feature image is used as a second training sample to obtain a plurality of second training samples, and the classification module is trained using the plurality of second training samples.

[0022] The present invention also provides a lightweight object detection device based on incremental learning, comprising:

[0023] A first processing module is configured to obtain an image to be tested and a target feature category image, wherein the target feature category image is an image corresponding to an untrained feature category in the image to be tested;

[0024] a second processing module, configured to input the image to be tested into a detection module, and obtain a plurality of sub-feature images output by the detection module;

[0025] a third processing module, configured to input the plurality of sub-feature images into a classification module, obtain feature expressions corresponding to the sub-feature images output by the classification module, and obtain a category corresponding to a first target sub-feature image among the plurality of sub-feature images; the category corresponding to the first target sub-feature image being a trained feature category;

[0026] a fourth processing module, configured to determine a category corresponding to a second target sub-feature image among the plurality of sub-feature images based on a similarity between a feature expression corresponding to the second target sub-feature image and the target feature category image; the category corresponding to the second target sub-feature image being an untrained feature category;

[0027] The output end of the detection module is connected to the input end of the classification module, the detection module is obtained after training based on sample images with sample sub-feature image labels, and the classification module is obtained after training based on sample sub-feature images with sample category labels.

[0028] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the lightweight object detection method based on incremental learning as described above is implemented.

[0029] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described lightweight object detection methods based on incremental learning.

[0030] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-described lightweight object detection methods based on incremental learning.

[0031] The lightweight object detection method and device based on incremental learning provided by the present invention adopt a two-stage object detection method based on incremental learning. In the detection stage, the detection module learns the target detection task that is independent of the category to distinguish foreground features from background features, thereby realizing the extraction of foreground features of the image to be tested; then, based on the new category in the image to be tested, a small number of target feature category images are provided to the classification module, so that the classification module can complete the recognition of the new category without repeated training, thereby significantly improving the generalization ability and scalability of the model, and is suitable for scenarios involving constantly changing information flows such as security detection, autonomous driving, and medical imaging. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0033] Figure 1 This is one of the flow charts of the lightweight object detection method based on incremental learning provided by the present invention;

[0034] Figure 2 This is the second flow chart of the lightweight object detection method based on incremental learning provided by the present invention;

[0035] Figure 3 This is the third flow chart of the lightweight object detection method based on incremental learning provided by the present invention;

[0036] Figure 4 This is one of the model structure diagrams of the lightweight object detection method based on incremental learning provided by the present invention;

[0037] Figure 5 This is the second model structure diagram of the lightweight object detection method based on incremental learning provided by the present invention;

[0038] Figure 6 This is the third model structure diagram of the lightweight object detection method based on incremental learning provided by the present invention;

[0039] Figure 7 1 is a schematic diagram of the detection results of the lightweight object detection method based on incremental learning provided by the present invention;

[0040] Figure 8 Schematic diagram of the principle of the lightweight object detection method based on incremental learning provided by the present invention;

[0041] Figure 9Schematic diagram of the structure of a lightweight object detection device based on incremental learning provided by the present invention;

[0042] Figure 10 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0043] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0044] The following combination Figures 1 to 8 The present invention describes a lightweight object detection method based on incremental learning.

[0045] The execution subject of the lightweight object detection method based on incremental learning of the present invention can be a lightweight object detection device based on incremental learning, or it can be a server, or it can also be a user's terminal, including but not limited to a mobile phone, tablet computer, and PC terminal.

[0046] like Figure 1 As shown, the lightweight object detection method based on incremental learning includes: step 110, step 120, step 130 and step 140.

[0047] Step 110: Acquire a test image and a target feature category image, where the target feature category image is an image corresponding to an untrained feature category in the test image;

[0048] In this step, the image to be tested is an image that needs to be subjected to image detection. The image to be tested may include one or more foreground features, and the categories of the foreground features may be the same or different.

[0049] The image to be measured may be an image or video frame retrieved from a database, or may be an image or video frame pulled from a network, or may be an image or video frame collected by a sensor, and the present invention is not limited thereto.

[0050] It is understandable that, in actual implementation, when a neural network model is used to perform image detection on an image to be tested, there may be a situation where the type of one or more foreground features in the image to be tested is a type that the neural network model has not been trained for.

[0051] The target feature category is the untrained category among the types corresponding to all foreground features of the image to be tested.

[0052] The number of target feature categories can be one or more.

[0053] The target feature category image is an image including target feature categories, and one target feature category image corresponds to one type of target feature category.

[0054] The target feature category image is determined and obtained by the user based on the content of the image to be tested and the training status of the neural network model.

[0055] For example, for Figure 3 The foreground features corresponding to the image to be tested include: stapler, ruler, pen, turtle, etc.

[0056] Among them, categories such as stapler, ruler and pen are trained categories, and turtle is an untrained category; then the turtle is the target feature category, and the image corresponding to the turtle is the target feature category image.

[0057] During the actual execution process, the user can obtain the image corresponding to the turtle and label it with categories to generate the target feature category image.

[0058] Step 120: input the image to be tested into the detection module, and obtain multiple sub-feature images output by the detection module;

[0059] In this step, the sub-feature image is an image including foreground features, and each sub-feature image includes one foreground feature.

[0060] For example, the sub-feature image may be a rectangular image, and the size of the rectangular area covers or is slightly larger than the outer contour of the foreground feature.

[0061] In the present invention, continue to refer to Figure 3 , a neural network model can be used to execute the method provided by the present invention.

[0062] The neural network model includes: a detection module and a classification module, and the output end of the detection module is connected to the input end of the classification module.

[0063] The detection module includes a detector, which is used to extract the foreground features of the image to be tested. The input of the detection module is the image to be tested, and the output is one or more sub-feature images, each of which corresponds to a foreground feature.

[0064] In some embodiments, the detection module can also output the position information of the sub-feature image in the image to be tested. The position information can be expressed as a vector or array containing numerical values ​​such as (x, y), length and width, where x and y represent the horizontal and vertical coordinates of the center point of the sub-feature image in the image to be tested, respectively, and length and width represent the length and width of the sub-feature image.

[0065] It should be noted that the category of the foreground feature corresponding to the sub-feature image output by the detection module may be a category that the neural network model has learned (such as stapler, ruler and pen, etc.), or it may be a category that the neural network model has not learned (such as turtle).

[0066] It is understandable that the detection module needs to be trained before application. The detection module is obtained after training based on sample images with sample sub-feature image labels. The specific training process will be described in subsequent embodiments and will not be repeated here.

[0067] Step 130: Input the plurality of sub-feature images into a classification module, obtain feature expressions corresponding to the sub-feature images output by the classification module, and obtain a category corresponding to a first target sub-feature image among the plurality of sub-feature images, wherein the category corresponding to the first target sub-feature image is a trained feature category;

[0068] In this step, the feature expression corresponding to the sub-feature image can be expressed as a low-dimensional vector, and each foreground feature corresponds to a feature expression.

[0069] The feature expressions corresponding to foreground features of the same category are similar, while the feature expressions corresponding to foreground features of different categories are far apart.

[0070] The first target sub-feature image is a sub-feature image corresponding to a foreground feature corresponding to a trained category in the image to be tested.

[0071] Continue to refer Figure 3 The classification module is used to predict the category corresponding to the sub-feature image; the input of the classification module is the multiple sub-feature images output by the detection module, and the output is the feature expression corresponding to each sub-feature image and the category corresponding to the first target sub-feature image.

[0072] It should be noted that, during the actual execution process, the classification module may also output the predicted probability of the category corresponding to the first target sub-feature image.

[0073] The predicted probability is used to characterize the credibility of the category corresponding to the first target sub-feature image predicted by the classification module. The higher the predicted probability, the higher the credibility.

[0074] It is understandable that the classification module needs to be trained before application. The classification module is obtained after training based on sample sub-feature images with sample category labels. The specific training process will be described in subsequent embodiments and will not be repeated here.

[0075] In some embodiments, the detection module includes a first backbone network, the classification module includes a second backbone network, the first backbone network and the second backbone network have the same structure and share weight initialization.

[0076] In this embodiment, the backbone network is a partial structure in YOLOv5.

[0077] It can be understood that YOLOv5 includes: an input end, a backbone network, a Neck end, and a prediction end, wherein the input end, the backbone network, the Neck end, and the prediction end are connected in sequence.

[0078] The first backbone network and the second backbone network have the same structure and share weights.

[0079] In the actual implementation process, Figure 4 As shown in Figure 1, YOLOv5 can be used as the overall architecture of the detection module, including the first backbone network, the three-level feature pyramid and the deep supervision structure.

[0080] like Figure 5 As shown in FIG, for the classification module, a second backbone network consistent with the detection module can be used as the overall architecture of the classification module, and shared weight initialization can be used to cope with the cold start problem.

[0081] According to the lightweight object detection method based on incremental learning provided by an embodiment of the present invention, a backbone network with the same structure is set for the detection module and the classification module to reuse weights, thereby enhancing the connection between the two-stage models, making it possible to effectively deal with the cold start problem, thereby improving the overall performance of the neural network model.

[0082] like Figure 6 As shown, in some embodiments, the first backbone network and the second backbone network include GhostNet modules.

[0083] In this embodiment, the GhostNet module can be used to replace the traditional convolution module, compress the number of feature channels and the number of high-order feature channels of the ROI head to achieve model lightweighting, so as to reduce the number of model parameters and the amount of calculation.

[0084] After repeated experiments and comparisons by the inventors, the size of the replaced detection module is 0.64M, which is significantly smaller than the traditional neural network model, and the detection accuracy mAP@0.5 is 0.957, which effectively ensures the accuracy of the model.

[0085] According to the lightweight object detection method based on incremental learning provided by an embodiment of the present invention, by replacing the traditional convolution module with the GhostNet module, etc., the total parameter amount of the model including the detection module and the classification module can be compressed to 1M while ensuring the accuracy of the model, thereby significantly reducing the parameter amount and calculation amount of the model, realizing the lightweight model, so that it can be easily deployed on small terminal devices such as security inspection.

[0086] Step 140: Determine the category corresponding to the second target sub-feature image based on the similarity between the feature expression corresponding to the second target sub-feature image in the plurality of sub-feature images and the target feature category image; the category corresponding to the second target sub-feature image is an untrained feature category.

[0087] In this step, the second target sub-feature image is a sub-feature image corresponding to a foreground feature corresponding to an untrained category in the image to be tested.

[0088] In the actual implementation process, the metric learning method is used to realize the recognition of the category corresponding to the second target sub-feature image to deal with unknown categories.

[0089] By comparing the similarity between the target feature category image acquired in advance and the second target sub-feature image, it is determined whether the second target sub-feature image belongs to the category corresponding to the target feature category image.

[0090] For example, a cosine similarity determination method may be used.

[0091] The following takes cosine similarity as an example to illustrate how to implement this step.

[0092] In some embodiments, step 140 may include:

[0093] Perform data augmentation processing on the target feature category image to generate the average mapping feature of N mapping features in the mapping feature space; where N is a positive integer;

[0094] generating a similarity between the feature expression corresponding to the second target sub-feature image and the average mapping feature;

[0095] When the similarity exceeds the target threshold, the category of the second target feature image is determined as the target feature category.

[0096] In this embodiment, for each foreground feature of an unknown category (i.e., a foreground feature of an untrained category), such as a turtle, there is a corresponding target feature category image (i.e., a Novel image). The Novel image (the image corresponding to the turtle) is a manually annotated image with the corresponding category information.

[0097] Of course, in other embodiments, each foreground feature of an unknown category may correspond to a small number of images of target feature categories, such as 3 or 5 images. It is understood that the fewer the number of images, the lower the cost of manual labeling and the subsequent computational cost.

[0098] Continue to refer Figure 3, by inputting the novel image into the classification module, the classification module performs data enhancement on the novel image to obtain the average mapping feature of N mapping features in the mapping feature space; where N is a positive integer.

[0099] The cosine similarity calculation is performed on the feature expression corresponding to the second target sub-feature image output by the classification module and the average mapping feature generated by the mapping feature space. When the cosine similarity exceeds the target threshold, the category of the second target feature image is determined to be the target feature category (such as turtle).

[0100] It will be appreciated that in this embodiment, no training is required for foreground features of unknown categories in the image to be tested. Prior to detection, the classification module simply maps the novel image to a mapping feature space, generates an average mapping feature corresponding to the unknown category, and stores this average mapping feature. During the detection process, the unknown category is assigned a corresponding category by comparing the similarity between the sub-feature image corresponding to the unknown category in the image to be tested and the average mapping feature stored in the mapping feature space. This allows for rapid detection of unknown categories, effectively responding to continuous information flows, and possessing a high ability to respond to new categories.

[0101] It should be noted that in actual application, as the number of categories recognized by the classification module continues to increase, the average mapping features are continuously corrected through similar samples in the model's historical recognition, which can make the model's performance better and better during use. This process does not require retraining, which can not only weaken the dependence on the initial unknown categories, but also fit the data flow of real scenarios and the phenomenon of new categories being added, thereby significantly improving the model's generalization ability to adapt to new categories.

[0102] like Figure 7 The figure shows a result obtained after performing image detection on an image to be detected by the method of the present invention.

[0103] According to the lightweight object detection method based on incremental learning provided by an embodiment of the present invention, a two-stage object detection method based on incremental learning is used. In the detection stage, the detection module learns target detection tasks that are independent of the category to distinguish foreground features from background features, thereby realizing the extraction of foreground features of the image to be tested; then, based on the new category in the image to be tested, a small number of target feature category images are provided to the classification module, so that the classification module can complete the recognition of the new category without repeated training, thereby significantly improving the generalization ability and scalability of the model, and is suitable for scenarios involving constantly changing and updated information flows, such as security detection, autonomous driving, and medical imaging.

[0104] The following combination Figure 2 , the training process of the detection module and the classification module is explained.

[0105] In some embodiments, before step 110, the method may include:

[0106] Obtain multiple sample images and a sample sub-feature image label corresponding to each sample image;

[0107] Each sample image and the sample sub-feature image label corresponding to the sample image are taken as a first training sample to obtain a plurality of first training samples, and the detection module is trained using the plurality of first training samples.

[0108] In this embodiment, the sample sub-feature image label is the real feature corresponding to the foreground feature in the sample image.

[0109] During the training process of the detection module, the detection module is trained using the first training sample to train the detection module to distinguish foreground features from background features, thereby recalling the foreground features.

[0110] like Figure 2 As shown, the sample image is input into the detector of the detection module for foreground feature recognition. After obtaining the bounding box of each foreground feature, each foreground feature is cropped to obtain a small image with the foreground feature, namely the sample sub-feature image.

[0111] The obtained sample sub-feature images can be used as samples for training the classification module and input into the classification module.

[0112] In some embodiments, before step 110, the method may include:

[0113] Obtain multiple sample sub-feature images and a category label corresponding to each sample sub-feature image;

[0114] Each sample sub-feature image and the category label corresponding to the sample sub-feature image is used as a second training sample to obtain multiple second training samples, and the classification module is trained using the multiple second training samples.

[0115] In this embodiment, during the training process of the classification module, the classification module is trained using the second training sample, so as to train the classification module to perform category prediction of the sample sub-feature image, thereby outputting the corresponding category.

[0116] Continue to refer Figure 2 , the different sample sub-feature images generated by the detection module are uniformly input into the second backbone network in the classification module to obtain the mapping feature space, metric learning is performed in the mapping space, and then classification learning of known categories is performed through a classifier.

[0117] It can be understood that, in the mapping space, feature expressions corresponding to sample sub-feature images of the same category are clustered together, and feature expressions corresponding to sample sub-feature images of different categories are mutually exclusive, so as to achieve clustering processing.

[0118] In some embodiments, the first backbone network in the detection module and the second backbone network in the classification module may be set to the same backbone network for weight reuse to enhance the correlation between the two modules.

[0119] The implementation is described below through specific embodiments.

[0120] For example, when learning to classify 98 known categories, the classic metric learning method is triplet loss. However, this method only considers the distance between each positive and negative sample pair, and the boundary distance considered for each positive and negative sample pair is the same.

[0121] In the present invention, Figure 8 As shown in Figure 2, the circle loss metric learning method can be used to dynamically assign coefficients to each positive and negative sample pair to improve the accuracy of the model.

[0122] Circle Loss can be expressed as:

[0123]

[0124] in, represents Circle Loss, j represents the jth inter-class sample pair, L represents the number of inter-class sample pairs, γ represents the scaling factor, A non-negative integer weight factor representing the inter-class part, Represents the similarity of the j-th inter-class sample pair, Δ n Represents the threshold between classes, i represents the sample pair within the i-th class, K represents the number of sample pairs within the class, A non-negative integer weight factor representing the intra-class part, Represents the similarity of sample pairs within the i-th class, Represents the gap between the similarity of the sample pair within the i-th class and the threshold within the class, Δ p Characterize the threshold within the class.

[0125] During the training phase, a random cropping method was used to crop portions of the original image. Optimally, the expected ratio of the object resolution during training to the object resolution during testing was 1 for optimal results. A formula was used to derive the optimal object resolution for testing. Taking into account both model accuracy and inference time, the inventors selected a 320x320 image resolution for testing, achieving a classification accuracy of 0.8813.

[0126] According to the lightweight object detection method based on incremental learning provided by an embodiment of the present invention, the detection module learns category-independent target detection tasks to distinguish foreground features from background features, and adopts metric learning to enhance feature expression capabilities to cope with unknown categories in the classification stage. The model is trained and optimized on known categories so that the model can learn the mapping ability from image to feature space, and the algorithm has the ability to expand the detection of new categories. In this way, in subsequent use, the model can complete the recognition of new categories without repeated training, thereby improving the generalization ability and scalability of the model.

[0127] The lightweight object detection device based on incremental learning provided by the present invention is described below. The lightweight object detection device based on incremental learning described below and the lightweight object detection method based on incremental learning described above can refer to each other.

[0128] like Figure 9 As shown, the lightweight object detection device based on incremental learning includes: a first processing module 910 , a second processing module 920 , a third processing module 930 and a fourth processing module 940 .

[0129] The first processing module 910 is used to obtain a test image and a target feature category image, where the target feature category image is an image corresponding to an untrained feature category in the test image;

[0130] The second processing module 920 is used to input the image to be tested into the detection module and obtain multiple sub-feature images output by the detection module;

[0131] The third processing module 930 is configured to input the plurality of sub-feature images into the classification module, obtain the feature expressions corresponding to the sub-feature images output by the classification module, and obtain the category corresponding to the first target sub-feature image among the plurality of sub-feature images; the category corresponding to the first target sub-feature image is a trained feature category;

[0132] A fourth processing module 940 is configured to determine a category corresponding to the second target sub-feature image based on a similarity between a feature expression corresponding to the second target sub-feature image and the target feature category image among the plurality of sub-feature images; the category corresponding to the second target sub-feature image is an untrained feature category;

[0133] Among them, the output end of the detection module is connected to the input end of the classification module. The detection module is obtained after training based on the sample image with the sample sub-feature image label, and the classification module is obtained after training based on the sample sub-feature image with the sample category label.

[0134] According to the lightweight object detection device based on incremental learning provided by an embodiment of the present invention, a two-stage object detection method based on incremental learning is used. In the detection stage, the detection module learns target detection tasks that are independent of the category to distinguish foreground features from background features, thereby realizing the extraction of foreground features of the image to be tested; then, based on the new category in the image to be tested, a small number of target feature category images are provided to the classification module, so that the classification module can complete the recognition of the new category without repeated training, thereby significantly improving the generalization ability and scalability of the model, and is suitable for scenarios involving constantly changing information flows such as security detection, autonomous driving, and medical imaging.

[0135] In some embodiments, the fourth processing module 940 may also be used to:

[0136] Perform data augmentation processing on the target feature category image to generate the average mapping feature of N mapping features in the mapping feature space; where N is a positive integer;

[0137] generating a similarity between the feature expression corresponding to the second target sub-feature image and the average mapping feature;

[0138] When the similarity exceeds the target threshold, the category of the second target feature image is determined as the target feature category.

[0139] In some embodiments, the detection module includes a first backbone network, the classification module includes a second backbone network, the first backbone network and the second backbone network have the same structure and share weight initialization.

[0140] In some embodiments, the first backbone network and the second backbone network include GhostNet modules.

[0141] In some embodiments, the apparatus may further include a fifth processing module configured to:

[0142] Before obtaining the image to be tested and the target feature category image, a plurality of sample images and a sample sub-feature image label corresponding to each sample image are obtained;

[0143] Each sample image and the sample sub-feature image label corresponding to the sample image are taken as a first training sample to obtain a plurality of first training samples, and the detection module is trained using the plurality of first training samples.

[0144] In some embodiments, the apparatus may further include a sixth processing module configured to:

[0145] Before obtaining the image to be tested and the target feature category image, a plurality of sample sub-feature images and a category label corresponding to each sample sub-feature image are obtained;

[0146] Each sample sub-feature image and the category label corresponding to the sample sub-feature image is used as a second training sample to obtain multiple second training samples, and the classification module is trained using the multiple second training samples.

[0147] Figure 10 An example of a physical structure diagram of an electronic device is shown below. Figure 10 As shown, the electronic device may include: a processor (processor) 1010, a communication interface (Communications Interface) 1020, a memory (memory) 1030 and a communication bus 1040, wherein the processor 1010, the communication interface 1020, and the memory 1030 communicate with each other through the communication bus 1040. The processor 1010 can call the logic instructions in the memory 1030 to execute a lightweight object detection method based on incremental learning, which includes: obtaining a test image and a target feature category image, where the target feature category image is an image corresponding to an untrained feature category in the test image; inputting the test image into a detection module to obtain multiple sub-feature images output by the detection module; inputting the multiple sub-feature images into a classification module to obtain feature expressions corresponding to the sub-feature images output by the classification module, and a category corresponding to a first target sub-feature image in the multiple sub-feature images; the category corresponding to the first target sub-feature image is a trained feature category; based on the feature expression and target feature category image corresponding to a second target sub-feature image in the multiple sub-feature images, determining the category corresponding to the second target sub-feature image; the category corresponding to the second target sub-feature image is an untrained feature category; wherein the output end of the detection module is connected to the input end of the classification module, the detection module is obtained after training based on a sample image with a sample sub-feature image label, and the classification module is obtained after training based on a sample sub-feature image with a sample category label.

[0148] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0149] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the lightweight object detection method based on incremental learning provided by the above methods, and the method includes: obtaining a test image and a target feature category image, the target feature category image is an image corresponding to an untrained feature category in the test image; inputting the test image into a detection module to obtain a plurality of sub-feature images output by the detection module; inputting the plurality of sub-feature images into a classification module to obtain a plurality of sub-feature images output by the classification module. A feature expression corresponding to a sub-feature image and a category corresponding to a first target sub-feature image among multiple sub-feature images are output; the category corresponding to the first target sub-feature image is a trained feature category; based on the feature expression corresponding to a second target sub-feature image among multiple sub-feature images and the target feature category image, the category corresponding to the second target sub-feature image is determined; the category corresponding to the second target sub-feature image is an untrained feature category; wherein the output end of the detection module is connected to the input end of the classification module, the detection module is obtained after training based on a sample image with a sample sub-feature image label, and the classification module is obtained after training based on a sample sub-feature image with a sample category label.

[0150] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned lightweight object detection method based on incremental learning, the method comprising: obtaining an image to be tested and a target feature category image, the target feature category image being an image corresponding to an untrained feature category in the image to be tested; inputting the image to be tested into a detection module to obtain a plurality of sub-feature images output by the detection module; inputting the plurality of sub-feature images into a classification module to obtain a feature expression corresponding to the sub-feature image output by the classification module, and a category corresponding to a first target sub-feature image in the plurality of sub-feature images; the category corresponding to the first target sub-feature image is a trained feature category; determining the category corresponding to the second target sub-feature image based on the feature expression corresponding to the second target sub-feature image in the plurality of sub-feature images and the target feature category image; the category corresponding to the second target sub-feature image is an untrained feature category; wherein the output end of the detection module is connected to the input end of the classification module, the detection module is obtained after training based on a sample image with a sample sub-feature image label, and the classification module is obtained after training based on a sample sub-feature image with a sample category label.

[0151] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0152] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0153] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A lightweight object detection method based on incremental learning, characterized in that: include: Acquire an image to be tested and a target feature category image, wherein the target feature category image is an image corresponding to an untrained feature category in the image to be tested; Inputting the image to be tested into a detection module, and obtaining a plurality of sub-feature images output by the detection module; Inputting the plurality of sub-feature images into a classification module, obtaining feature expressions corresponding to the sub-feature images output by the classification module, and a category corresponding to a first target sub-feature image among the plurality of sub-feature images; the category corresponding to the first target sub-feature image is a trained feature category; determining a category corresponding to a second target sub-feature image based on a similarity between a feature expression corresponding to a second target sub-feature image in the plurality of sub-feature images and the target feature category image; the category corresponding to the second target sub-feature image being an untrained feature category; The output end of the detection module is connected to the input end of the classification module, the detection module is obtained after training based on sample images with sample sub-feature image labels, and the classification module is obtained after training based on sample sub-feature images with sample category labels.

2. The lightweight object detection method based on incremental learning according to claim 1, characterized in that The determining, based on the similarity between the feature expression corresponding to the second target sub-feature image in the plurality of sub-feature images and the target feature category image, the category corresponding to the second target sub-feature image includes: Performing data enhancement processing on the target feature category image to generate an average mapping feature of N mapping features in a mapping feature space; wherein N is a positive integer; generating a similarity between a feature expression corresponding to the second target sub-feature image and the average mapping feature; When the similarity exceeds a target threshold, the category of the second target sub-feature image is determined as the target feature category.

3. The lightweight object detection method based on incremental learning according to claim 1, characterized in that The detection module includes a first backbone network, and the classification module includes a second backbone network. The first backbone network and the second backbone network have the same structure and share weights.

4. The lightweight object detection method based on incremental learning according to claim 3, characterized in that The first backbone network and the second backbone network include GhostNet modules.

5. The lightweight object detection method based on incremental learning according to any one of claims 1 to 4, characterized in that: Before acquiring the image to be measured and the target feature category image, the method includes: Acquire multiple sample images and a sample sub-feature image label corresponding to each of the sample images; Each of the sample images and the sample sub-feature image label corresponding to the sample image is taken as a first training sample to obtain a plurality of first training samples, and the detection module is trained using the plurality of first training samples.

6. The lightweight object detection method based on incremental learning according to any one of claims 1 to 4, characterized in that: Before acquiring the image to be measured and the target feature category image, the method includes: Obtaining a plurality of sample sub-feature images and a category label corresponding to each of the sample sub-feature images; Each of the sample sub-feature images and the category label corresponding to the sample sub-feature image is used as a second training sample to obtain a plurality of second training samples, and the classification module is trained using the plurality of second training samples.

7. A lightweight object detection device based on incremental learning, characterized in that: include: A first processing module is configured to obtain an image to be tested and a target feature category image, wherein the target feature category image is an image corresponding to an untrained feature category in the image to be tested; a second processing module, configured to input the image to be tested into a detection module, and obtain a plurality of sub-feature images output by the detection module; a third processing module, configured to input the plurality of sub-feature images into a classification module, obtain feature expressions corresponding to the sub-feature images output by the classification module, and obtain a category corresponding to a first target sub-feature image among the plurality of sub-feature images; the category corresponding to the first target sub-feature image being a trained feature category; a fourth processing module, configured to determine a category corresponding to a second target sub-feature image among the plurality of sub-feature images based on a similarity between a feature expression corresponding to the second target sub-feature image and the target feature category image; the category corresponding to the second target sub-feature image being an untrained feature category; The output end of the detection module is connected to the input end of the classification module, the detection module is obtained after training based on sample images with sample sub-feature image labels, and the classification module is obtained after training based on sample sub-feature images with sample category labels.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the lightweight object detection method based on incremental learning as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the lightweight object detection method based on incremental learning according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the lightweight object detection method based on incremental learning according to any one of claims 1 to 6 is implemented.

Images