Small sample target detection method and device based on mixed experts

Abstract

The invention discloses a small sample target detection method and device based on hybrid experts, and belongs to the technical field of target detection, and the method comprises the steps: employing a repeated up-sampling and data enhancement mode to carry out the enhancement of a new-class small sample image; constructing a small sample target detection model based on the hybrid experts, wherein the small sample target detection model comprises a shared shallow feature network, a unique deep feature network of each hybrid expert, a feature fusion network and a target detection network corresponding to each hybrid expert; and performing parameter optimization on the small sample target detection model based on the hybrid experts by using the basic category image and the new category enhanced image, and learning knowledge of different depths by using the hybrid experts and skillfully combining features of different depths, thereby improving the detection performance of the model on a small sample target detection task. And meanwhile, the combination of up-sampling and data enhancement is proposed, so that the overfitting problem caused by the up-sampling process is effectively alleviated, the diversity of small sample data is enhanced, and the robustness of the model is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a small sample target detection method and device based on mixed experts. Background Art

[0002] Object detection is a basic task in the field of computer vision, which aims to identify various objects in an image and classify them. Traditional object detection methods often rely on a large amount of labeled data to train the model, but it is expensive to collect a large amount of high-quality labeled data, so Few-Shot Object Detection (FSOD) came into being. Few-Shot Object Detection aims to solve the problem of performance degradation of traditional object detection models when labeled data is scarce, narrow the gap between machine vision models and human visual systems, and enable the model to accurately detect new categories of objects with only a very small number of labeled samples.

[0003] The current small sample target detection methods mainly improve the detection performance of the model from three aspects. First, improve the forgetfulness of the model: the model needs to use the basic category knowledge learned during the pre-training period to assist in the learning of small sample categories. Therefore, it is extremely important not to forget the previously learned knowledge when learning small sample categories; second, improve the confusion of the model: when learning small sample categories, the model does not want to be confused with the basic category knowledge learned during the pre-training period, resulting in an inability to distinguish between basic categories and small sample categories; third, category imbalance processing: since the number of basic category images of the model during the pre-training period is huge, it is particularly important to balance the huge difference in the number of small sample categories and basic category images.

[0004] Existing small sample target detection methods are usually divided into two stages. Stage 1 is pre-trained on basic category images with a large amount of data annotations, and then stage 2 freezes the weights of the backbone network (ResNet-101 is used in this paper) and fine-tunes on small sample categories and basic categories using various schemes. The disadvantage of this method is that the features of the backbone network have been frozen during fine-tuning and cannot be further optimized. At the same time, only the deepest features are used for learning. The learned features sometimes cannot distinguish between basic categories and new categories well. Although there are already feature pyramid networks (FPN) to deal with the problem of using only the deepest network, FPN is not flexible enough in dealing with different depth features, so the problem of combining different depth features is still worth exploring; in addition, when dealing with category imbalance, small sample target detection tasks often use simple upsampling or downsampling methods, thereby ignoring more traditional data enhancement methods. Summary of the invention

[0005] In view of the above, the purpose of the present invention is to provide a small sample target detection method and device based on hybrid experts. By introducing multiple hybrid experts, not only can their respective deep features be obtained, but also shallow features of different depths can be obtained respectively. By combining deep features and shallow features and using knowledge distillation to allow each hybrid expert to learn from each other, the target detection model can capture the key points of classification from different deep features and achieve better classification effects.

[0006] To achieve the above-mentioned object of the invention, an embodiment provides a small sample target method based on mixed experts, comprising the following steps: Use base category images to perform preliminary training on the object detection model; Repeated upsampling and data enhancement are used to enhance the small sample images of the new category to obtain enhanced images of the new category; A small sample target detection model based on hybrid experts is constructed, which includes a shared shallow feature network, a unique deep feature network of each hybrid expert, a feature fusion network, and a target detection network corresponding to each hybrid expert, wherein the shared shallow feature network is used to extract shallow features of different depths from the input image, the unique deep feature network is used to extract respective deep features based on the shallow features, the feature fusion network is used to fuse the shallow features of different depths obtained by each hybrid expert and the respective deep features to obtain respective fused features, and the target detection network is used to predict the target detection result corresponding to each hybrid expert based on the fused features; wherein the shared shallow feature network and the unique deep feature network of each hybrid expert are initialized using the backbone network of the initially trained target detection model; The parameters of the small sample target detection model based on mixed experts are optimized using the base category images and the new category enhanced images. The loss function used in the parameter optimization includes the target detection loss corresponding to each mixed expert and the distillation loss between each mixed expert. A small sample target detection model with optimized parameters is used to perform small sample target detection, and the final target detection result is screened from the target detection results of multiple mixed experts.

[0007] Preferably, the target detection model adopts the Faster RCNN network. When performing preliminary training on the Faster RCNN network, a cross entropy loss is used for the classification results included in the target detection results, and a border regression loss is used for the border regression results included in the target detection results.

[0008] Preferably, the new category small sample image is enhanced by repeated upsampling and data enhancement to obtain the new category enhanced image, including: First, the small sample images of the new category are repeatedly upsampled to increase the number of images, and then data augmentation is performed on each small sample image set with increased number, where the data augmentation methods include contrast adjustment, saturation adjustment, hue adjustment, and the addition of Gaussian noise and salt and pepper noise.

[0009] Preferably, the feature fusion network fuses the shallow features of different depths obtained by each hybrid expert with their respective deep features to obtain respective fused features, including: First, the shallow features of different depths obtained by each hybrid expert are downsampled to the same size as the deep features, and then the downsampled shallow features are fused with the deep features to obtain the fused features. ,in, Indicates i The shallow features obtained by the mixed experts, Indicates i The deep features corresponding to the mixed experts, The symbol represents the Hadamard product, represents downsampling processing, Represents the budget for taking logarithms.

[0010] Preferably, the target detection network corresponding to each hybrid expert includes a region candidate network, and an ROI Head module that provides category prediction and bounding box regression prediction for each candidate region. The classification results and bounding box regression results output by the ROI Head module constitute the target detection results corresponding to each hybrid expert.

[0011] Preferably, the loss function used for parameter optimization is Contains the target detection loss corresponding to each hybrid expert , and distillation losses between mixing experts , respectively expressed as: ; ; ; in, represents the sum of the classification losses corresponding to all hybrid experts, represents the sum of the logistic regression losses corresponding to all mixed experts, and represents the hyperparameter, and represents a mixed expert index, m represents the number of mixed experts, and Respectively represent i A mixed expert and j The classification results of the mixed experts are express and The KL divergence loss between .

[0012] Preferably, the final target detection result is selected from the target detection results of multiple mixed experts, including: The target detection result corresponding to the hybrid expert with the highest category score is selected from the target detection results of multiple hybrid experts as the final target detection result.

[0013] To achieve the above invention, an embodiment of the present invention further provides a small sample target detection device based on mixed experts, comprising: A preliminary training module, which is used to perform preliminary training on the target detection model using basic category images; A data enhancement module is used to enhance a small sample image of a new category by repeated upsampling and data enhancement to obtain an enhanced image of a new category; A model building module, which is used to build a small sample target detection model based on mixed experts, which includes a shared shallow feature network, a unique deep feature network of each mixed expert, a feature fusion network, and a target detection network corresponding to each mixed expert, wherein the shared shallow feature network is used to extract shallow features of different depths from the input image, the unique deep feature network is used to extract respective deep features based on the shallow features, the feature fusion network is used to fuse the shallow features of different depths obtained by each mixed expert and the respective deep features to obtain respective fused features, and the target detection network is used to predict the target detection result corresponding to each mixed expert based on the fused features; wherein the shared shallow feature network and the unique deep feature network of each mixed expert are initialized using the backbone network of the preliminarily trained target detection model; A model training module, which is used to optimize the parameters of the small sample target detection model based on mixed experts using the base category images and the new category enhanced images. The loss function used in the parameter optimization includes the target detection loss corresponding to each mixed expert and the distillation loss between each mixed expert. The target detection module is used to perform small sample target detection using a small sample target detection model with optimized parameters, and to filter the final target detection result from the target detection results of multiple mixed experts.

[0014] To achieve the above-mentioned purpose of the invention, an embodiment further provides a computing device, including a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the above-mentioned small sample target detection method based on mixed experts.

[0015] To achieve the above-mentioned purpose of the invention, an embodiment further provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the above-mentioned small sample target detection method based on mixed experts is implemented.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The method of the present invention is the first method to use hybrid experts in the field of small sample target detection. It not only avoids the negative impact of freezing the parameters of the backbone network, but also can more reasonably combine features of different depths to more comprehensively analyze the image and optimize the image classification process. The method of the present invention is the first method that combines repeated upsampling and data enhancement in the field of small sample target detection. It not only alleviates the overfitting problem that may be caused by the simple upsampling process, but also increases the amount of data in the small sample category, so that the model can cope with a variety of image detection problems, thereby improving the generalization of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 is a flow chart of a small sample target detection method based on mixed experts provided in an embodiment; Figure 2 is a schematic diagram of the structure of the preliminary training of the small sample target detection model provided in the embodiment; Figure 3 is a schematic diagram of data enhancement provided by an embodiment; Figure 4 is a schematic diagram of the structure of a small sample target detection model based on mixed experts provided in an embodiment; Figure 5 It is a schematic diagram of the structure of a small sample target detection device based on mixed experts provided in an embodiment. DETAILED DESCRIPTION

[0019] To make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0020] The inventive concept of the present invention is: the embodiment of the present invention provides a small sample target detection method and device based on hybrid experts. To address the technical problems of freezing the backbone network in existing small sample target detection and being unable to flexibly use different deep features to distinguish basic categories from new categories, by introducing multiple hybrid experts, not only can their respective deep features be obtained, but also shallow features of different depths can be obtained respectively. By combining deep features and shallow features and using knowledge distillation to allow each hybrid expert to learn from each other, the target detection model can capture the key points of classification from different deep features to achieve better classification effects. To address the technical problem of category imbalance in small samples, repeated upsampling and data enhancement are used to expand data to improve the robustness of the target detection model.

[0021] like Figure 1 As shown, the embodiment provides a small sample target detection method based on mixed experts, comprising the following steps: S110, performing preliminary training on the target detection model using basic category images.

[0022] In the embodiment, Figure 2 As shown in the figure, the small sample target detection model can use the Faster RCNN network, which includes a backbone network for extracting features, a Region Proposal Network (RPN module) for selecting candidate regions, and an ROI Head module that provides category prediction and bounding box regression prediction for each candidate region. The ROI Head module includes ROI pooling, category classifier, and bounding box regression. This target detection model is used for target detection in basic category images. A large number of labeled basic category images are used to perform preliminary training on the target detection model. During specific training, the cross entropy loss is used for the classification results contained in the target detection results, and the bounding box regression loss is used for the bounding box regression results contained in the target detection results.

[0023] S120, enhancing the small sample image of the new category by repeated upsampling and data enhancement to obtain an enhanced image of the new category.

[0024] In the data enhancement process of the embodiment, three traditional technologies are applied to enrich the diversity of training data and prevent model overfitting, including horizontal flipping, rotation and brightness adjustment. Horizontal flipping increases the sample size by symmetrically flipping the image along its vertical central axis; rotation enhances the model's recognition ability of multi-view objects by randomly rotating the image by a fixed angle; brightness adjustment enhances the model's adaptability to various lighting conditions by randomly changing the image brightness. In order to further improve the diversity of data and the robustness of the model, the embodiment also introduces five other enhancement technologies, including contrast adjustment, saturation adjustment, hue adjustment, Gaussian noise and salt and pepper noise addition. Contrast adjustment highlights the key features in the image by enhancing the difference between light and dark in the image; saturation adjustment makes the color of the image more vivid, helping the model learn more color features; hue adjustment simulates image changes at different color temperatures by changing the hue of the image; Gaussian noise improves the model's adaptability to noisy data by simulating noise interference in different environments; salt and pepper noise enhances the model's anti-interference ability to image noise by adding noise points. The effects of each are as follows: Figure 3 In practical applications, each enhancement scheme will be applied to the specified small sample category image with a certain probability, and the enhancement process can be superimposed.

[0025] In response to the problem of imbalanced categories, there is a simple but effective Repeated Factor Sampling (RFS) method in past inventions to upsample new category samples. RFS alleviates the extreme imbalance in the distribution of category data by repeatedly sampling certain categories of data (i.e., increasing the number of occurrences of category samples), thereby improving the model's learning ability for categories with few samples. In the method of the present invention, the idea of ​​upsampling is adopted, and the data enhancement strategy is applied to repeated sampling, which not only solves the problem of unbalanced data distribution and alleviates the forgetting of data, but also these data enhancement operations not only improve the training effect and generalization ability of the model, but also enhance the model's adaptability to complex environments, providing a solid foundation for subsequent model training and optimization. The use of data enhancement technology effectively improves the recognition accuracy of the model under multiple viewing angles, lighting conditions and noise interference, demonstrating its key role in improving the robustness and performance of the model.

[0026] Based on this, when performing data enhancement, the embodiment first repeatedly upsamples the small sample images of the new category to increase the number of images, and then performs data enhancement on each small sample image set with the increased number.

[0027] S130, constructing a small sample target detection model based on hybrid experts, which includes a shared shallow feature network, a unique deep feature network for each hybrid expert, a feature fusion network, and a target detection network corresponding to each hybrid expert, wherein the shared shallow feature network and the unique deep feature network for each hybrid expert are initialized using the backbone network of the initially trained target detection model.

[0028] With regard to the above-mentioned technical problems of freezing the weights of the backbone network and being unable to reasonably utilize different deep features, an embodiment of the present invention proposes to use a hybrid expert network to solve this technical problem. First, since repeated upsampling and data enhancement methods are proposed to address the problem of category imbalance, this can obtain many different images of new categories. Therefore, blindly freezing the weights of the backbone network will make it impossible to train this part of the images well. Therefore, it is imperative to restore the trainability of the backbone network. At the same time, in order to enable the network to learn more comprehensive features, an embodiment of the present invention adopts a hybrid expert method to integrate shallow features into deep features for joint learning.

[0029] like Figure 4 As shown, the small sample target detection model constructed based on hybrid experts includes a shared shallow feature network, a unique deep feature network for each hybrid expert, a feature fusion network, and a target detection network corresponding to each hybrid expert, wherein the shared shallow feature network is used to extract shallow features of different depths from the input image, the unique deep feature network is used to extract respective deep features based on the shallow features, the feature fusion network is used to fuse the shallow features of different depths obtained by each hybrid expert with their respective deep features to obtain respective fused features, and the target detection network is used to predict the target detection results corresponding to each hybrid expert based on the fused features.

[0030] In the above model, each hybrid expert shares the first few layers of the backbone network (such as ResNet-101) in the target detection model as a shared shallow feature network, and independently owns the last few layers as a unique deep feature network. The parameters of the shared part and the independent part are initialized using the parameters of the backbone network in the pre-training stage. This is because the model of the present invention is mainly optimized for small sample categories. In comparison, the attention of the basic category will decrease. Therefore, if the multi-expert network is completely retrained, it will affect the basic category. Therefore, using the pre-trained backbone network weights on the basic category for initialization can effectively avoid this problem. Subsequently, feature fusion is performed through the feature fusion network. Specifically, each hybrid expert will obtain shallow features of different depths from different parts of the shared shallow feature network. and the deep features obtained by the unique deep feature network Deep fusion, Representative expert number, . Since the size of shallow features and deep features are different and cannot be directly connected or calculated, a convolution layer of corresponding size is designed after each shallow feature to perform downsampling processing to achieve alignment of shallow features and deep features. Then, the shallow features are fused with the deep features using the Hadamard product, and the features are converted into logarithms to prevent gradient explosion and other problems caused by excessive data, and the final fusion features of each hybrid expert are obtained. ,in, Indicates i The shallow features obtained by the mixed experts, Indicates i The deep features corresponding to the mixed experts, The symbol represents the Hadamard product, represents downsampling processing, Represents the logarithmic budget. Since the fusion features obtained by each hybrid expert are the fusion of shallow features of different depths and their own deep features, the subsequent target detection network of each hybrid expert is also exclusive. The target detection network corresponding to each hybrid expert includes a region proposal network (RPN), and an ROI Head module that provides category prediction and bounding box regression prediction for each candidate region. The classification results and bounding box regression results output by the ROI Head module constitute the target detection results corresponding to each hybrid expert.

[0031] S140, using the base category image and the new category enhanced image to optimize the parameters of the small sample target detection model based on the mixed experts, the loss function used in the parameter optimization includes the target detection loss corresponding to each mixed expert and the distillation loss between each mixed expert.

[0032] In the embodiment, in order to better utilize features of different depths to promote category prediction, knowledge distillation is performed between any two hybrid experts so that they can learn from each other. Since each hybrid expert has the same architecture at the deepest position of the network, it can be ensured that each hybrid expert can play the role of a teacher or a student. Any two hybrid experts can perform mutual knowledge distillation, providing each expert with a perfect opportunity to gather knowledge of different depths. In the process of training the small sample target detection model, KL divergence is used as the distillation loss to allow different hybrid experts to perform mutual knowledge distillation learning. KL divergence can quantify the difference between probability distributions and help adjust model parameters to make the distribution generated by the model closer to the target distribution. KL divergence can prompt each hybrid expert to learn knowledge of different depths. Since each hybrid expert has learned features of different depths, the information obtained is more comprehensive, so the difference between the basic category and the new category will be clearer. Therefore, the learning method of the hybrid expert can improve the confusion between categories. In addition, each hybrid expert also needs to learn the true value (Ground Truth) of the real label, which can ensure that each hybrid expert will not deviate from the original goal. Here, the traditional target detection loss can be maintained. The loss function used for parameter optimization Contains the target detection loss corresponding to each hybrid expert , and distillation losses between mixing experts , respectively expressed as: ; ; ; in, Represents the sum of the classification losses corresponding to all hybrid experts. The classification loss can be the cross entropy loss. represents the sum of the logistic regression losses corresponding to all mixed experts, and represents the hyperparameter, and represents a mixed expert index, m represents the number of mixed experts, and Respectively represent i A mixed expert and j The classification results of the mixed experts are express and The KL divergence loss between .

[0033] Through the above loss function, the model can fully learn features of different depths.

[0034] S150, performing small sample target detection using a small sample target detection model with optimized parameters, and screening a final target detection result from target detection results of multiple mixed experts.

[0035] In the inference stage, multiple hybrid experts in the small sample target detection model are used to simultaneously detect targets on the image. Since multiple hybrid experts can make predictions in parallel, the inference time will not be increased too much. When the target detection results of multiple experts are obtained, the target detection results include logistic regression results and classification results. ,in k is the total number of base categories and new categories, Represents the classification result of the i-th hybrid expert for the k-th category. The target detection result of the hybrid expert with the highest category score is selected as the final target result. This is because the higher the score means the higher the confidence and the closer the image feature is to the center of the category. Therefore, this further strengthens the compactness within the class and alleviates the forgetfulness in small sample target detection. The specific expression is as follows: ; Among them, max refers to the maximum value, argmax refers to the position of the maximum value, and id refers to the number of the hybrid expert finally selected.

[0036] The small sample target detection method based on hybrid experts provided in the above embodiment uses hybrid experts to cleverly combine features of different depths, learn knowledge of different depths, and improve the detection performance of the model in the small sample target detection task; at the same time, a method combining repeated upsampling and data enhancement is proposed, which not only effectively alleviates the overfitting problem caused by the upsampling process, but also enhances the diversity of small sample data and improves the robustness of the model.

[0037] The core of the small sample target detection method based on hybrid experts provided in the above embodiment is to use the hybrid expert method and combine the repeated upsampling and data enhancement methods to solve a series of problems in the field of small sample target detection. The present invention is not limited to the above-mentioned backbone network, target detection loss and distillation loss. Simply replacing the backbone network, simply changing the KL divergence and cross entropy loss function, or simply increasing or deleting the number of hybrid experts are all within the protection scope of the present invention.

[0038] like Figure 5As shown, the embodiment also provides a small sample target detection device 500 based on hybrid experts, including: a preliminary training module 510, a data enhancement module 520, a model building module 530, a model training module 540, and a target detection module 550, wherein the preliminary training module 510 is used to perform preliminary training on the target detection model using the basic category image; the data enhancement module 520 is used to enhance the new category small sample image by repeated upsampling and data enhancement to obtain the new category enhanced image; the model building module 530 is used to construct a small sample target detection model based on hybrid experts, which includes a shared shallow feature network, a unique deep feature network of each hybrid expert, a feature fusion network, and a target detection network corresponding to each hybrid expert, wherein the shared shallow feature network is used to extract shallow features of different depths from the input image, and the unique deep feature network is used to extract shallow features based on the shallow features. The respective deep features, the feature fusion network is used to fuse the shallow features of different depths obtained by each hybrid expert and the respective deep features to obtain the respective fused features, and the target detection network is used to predict the target detection results corresponding to each hybrid expert based on the fused features; wherein, the shared shallow feature network and the unique deep feature network of each hybrid expert are initialized using the backbone network of the initially trained target detection model; the model training module 540 is used to optimize the parameters of the above-mentioned small sample target detection model based on hybrid experts using basic category images and new category enhanced images, and the loss function used in parameter optimization includes the target detection loss corresponding to each hybrid expert and the distillation loss between each hybrid expert; the target detection module 550 is used to perform small sample target detection using the parameter-optimized small sample target detection model, and screen the final target detection result from the target detection results of multiple hybrid experts.

[0039] It should be noted that the small sample target detection device based on mixed experts provided in the above embodiment should be illustrated by the division of the above functional modules when performing small sample target detection. The above functional distribution can be completed by different functional modules as needed, that is, the internal structure of the terminal or server is divided into different functional modules to complete all or part of the functions described above. In addition, the small sample target detection device based on mixed experts provided in the above embodiment belongs to the same concept as the small sample target detection method embodiment based on mixed experts. The specific implementation process is detailed in the small sample target detection method embodiment based on mixed experts, which will not be repeated here.

[0040] Based on the same inventive concept, an embodiment further provides a computing device, including a memory and one or more processors, wherein executable code is stored in the memory, and when the one or more processors execute the executable code, the device is used to implement the above-mentioned small sample target detection method based on mixed experts, which specifically includes the following steps: S110, performing preliminary training on the target detection model using the basic category images; S120, enhancing the small sample image of the new category by repeated upsampling and data enhancement to obtain an enhanced image of the new category; S130, constructing a small sample target detection model based on hybrid experts, which includes a shared shallow feature network, a unique deep feature network of each hybrid expert, a feature fusion network, and a target detection network corresponding to each hybrid expert, wherein the shared shallow feature network is used to extract shallow features of different depths from the input image, the unique deep feature network is used to extract respective deep features based on the shallow features, the feature fusion network is used to fuse the shallow features of different depths obtained by each hybrid expert and the respective deep features to obtain respective fused features, and the target detection network is used to predict the target detection result corresponding to each hybrid expert based on the fused features; wherein the shared shallow feature network and the unique deep feature network of each hybrid expert are initialized using the backbone network of the initially trained target detection model; S140, optimizing parameters of the small sample target detection model based on mixed experts using the base category image and the new category enhanced image, wherein the loss function used in the parameter optimization includes the target detection loss corresponding to each mixed expert and the distillation loss between each mixed expert; S150, performing small sample target detection using a small sample target detection model with optimized parameters, and screening a final target detection result from target detection results of multiple mixed experts.

[0041] The computing device provided in the embodiment, in addition to the processor and memory, also includes hardware required for other services such as internal bus, network interface, memory, etc. at the hardware level. The memory is a non-volatile memory, and the processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the small sample target detection method based on hybrid experts described in S110-S150 above. Of course, in addition to the software implementation, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0042] Based on the same inventive concept, an embodiment further provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the above-mentioned small sample target detection method based on mixed experts is implemented, which specifically includes the following steps: S110, performing preliminary training on the target detection model using the basic category images; S120, enhancing the small sample image of the new category by repeated upsampling and data enhancement to obtain an enhanced image of the new category; S130, constructing a small sample target detection model based on hybrid experts, which includes a shared shallow feature network, a unique deep feature network of each hybrid expert, a feature fusion network, and a target detection network corresponding to each hybrid expert, wherein the shared shallow feature network is used to extract shallow features of different depths from the input image, the unique deep feature network is used to extract respective deep features based on the shallow features, the feature fusion network is used to fuse the shallow features of different depths obtained by each hybrid expert and the respective deep features to obtain respective fused features, and the target detection network is used to predict the target detection result corresponding to each hybrid expert based on the fused features; wherein the shared shallow feature network and the unique deep feature network of each hybrid expert are initialized using the backbone network of the initially trained target detection model; S140, optimizing parameters of the small sample target detection model based on mixed experts using the base category image and the new category enhanced image, wherein the loss function used in the parameter optimization includes the target detection loss corresponding to each mixed expert and the distillation loss between each mixed expert; S150, performing small sample target detection using a small sample target detection model with optimized parameters, and screening a final target detection result from target detection results of multiple mixed experts.

[0043] In the embodiment, computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data.

[0044] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A small sample target detection method based on mixed experts, characterized in that: The following steps are involved: Use base category images to perform preliminary training on the object detection model; Repeated upsampling and data enhancement are used to enhance the small sample images of the new category to obtain enhanced images of the new category; A small sample target detection model based on hybrid experts is constructed, which includes a shared shallow feature network, a unique deep feature network of each hybrid expert, a feature fusion network, and a target detection network corresponding to each hybrid expert, wherein the shared shallow feature network is used to extract shallow features of different depths from the input image, the unique deep feature network is used to extract respective deep features based on the shallow features, the feature fusion network is used to fuse the shallow features of different depths obtained by each hybrid expert and the respective deep features to obtain respective fused features, and the target detection network is used to predict the target detection result corresponding to each hybrid expert based on the fused features; wherein the shared shallow feature network and the unique deep feature network of each hybrid expert are initialized using the backbone network of the initially trained target detection model; The parameters of the small sample target detection model based on mixed experts are optimized using the base category images and the new category enhanced images. The loss function used in the parameter optimization includes the target detection loss corresponding to each mixed expert and the distillation loss between each mixed expert. A small sample target detection model with optimized parameters is used to perform small sample target detection, and the final target detection result is screened from the target detection results of multiple mixed experts.

2. The small sample target detection method based on mixed experts according to claim 1 is characterized in that: The target detection model adopts the Faster RCNN network. During the preliminary training of the Faster RCNN network, the cross entropy loss is used for the classification results contained in the target detection results, and the frame regression loss is used for the frame regression results contained in the target detection results.

3. The small sample target detection method based on mixed experts according to claim 1 is characterized in that: Repeated upsampling and data enhancement are used to enhance the small sample images of new categories to obtain enhanced images of new categories, including: First, the small sample images of the new category are repeatedly upsampled to increase the number of images, and then data augmentation is performed on each small sample image set with increased number, where the data augmentation methods include horizontal flipping, rotation, brightness adjustment, contrast adjustment, saturation adjustment, hue adjustment, and the addition of Gaussian noise and salt and pepper noise.

4. The small sample target detection method based on mixed experts according to claim 1 is characterized in that: Feature Fusion The network fuses the shallow features of different depths obtained by each hybrid expert with their respective deep features to obtain their respective fused features, including: First, the shallow features of different depths obtained by each hybrid expert are downsampled to the same size as the deep features, and then the downsampled shallow features are fused with the deep features to obtain the fused features. ,in, Indicates i The shallow features obtained by the mixed experts, Indicates i The deep features corresponding to the mixed experts, The symbol represents the Hadamard product, represents downsampling processing, Represents the budget for taking logarithms.

5. The small sample target detection method based on mixed experts according to claim 1 is characterized in that: The object detection network corresponding to each hybrid expert includes a region candidate network and an ROI Head module that provides category prediction and bounding box regression prediction for each candidate region. The classification results and bounding box regression results output by the ROI Head module constitute the object detection results corresponding to each hybrid expert.

6. The small sample target detection method based on mixed experts according to claim 1 is characterized in that: Loss function used for parameter optimization Contains the target detection loss corresponding to each hybrid expert , and distillation losses between mixing experts , respectively expressed as: ； ； ； in, represents the sum of the classification losses corresponding to all hybrid experts, represents the sum of the logistic regression losses corresponding to all mixed experts, and represents the hyperparameter, and represents a mixed expert index, m represents the number of mixed experts, and Respectively represent i A mixed expert and j The classification results of the mixed experts are express and The KL divergence loss between .

7. The small sample target detection method based on mixed experts according to claim 1 is characterized in that: The final target detection result is selected from the target detection results of multiple mixed experts, including: The target detection result corresponding to the hybrid expert with the highest category score is selected from the target detection results of multiple hybrid experts as the final target detection result.

8. A small sample target detection device based on mixed experts, characterized in that: include: A preliminary training module, which is used to perform preliminary training on the target detection model using basic category images; A data enhancement module is used to enhance a small sample image of a new category by repeated upsampling and data enhancement to obtain an enhanced image of a new category; A model building module, which is used to build a small sample target detection model based on mixed experts, which includes a shared shallow feature network, a unique deep feature network of each mixed expert, a feature fusion network, and a target detection network corresponding to each mixed expert, wherein the shared shallow feature network is used to extract shallow features of different depths from the input image, the unique deep feature network is used to extract respective deep features based on the shallow features, the feature fusion network is used to fuse the shallow features of different depths obtained by each mixed expert and the respective deep features to obtain respective fused features, and the target detection network is used to predict the target detection result corresponding to each mixed expert based on the fused features; wherein the shared shallow feature network and the unique deep feature network of each mixed expert are initialized using the backbone network of the preliminarily trained target detection model; A model training module, which is used to optimize the parameters of the small sample target detection model based on mixed experts by using the base category images and the new category enhanced images. The loss function used in the parameter optimization includes the target detection loss corresponding to each mixed expert and the distillation loss between each mixed expert. The target detection module is used to perform small sample target detection using a small sample target detection model with optimized parameters, and to filter the final target detection result from the target detection results of multiple mixed experts.

9. A computing device comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the one or more processors execute the executable code, they are used to implement the small sample target detection method based on mixed experts as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the small sample target detection method based on mixed experts described in any one of claims 1 to 7 is implemented.

Images