Target detection method and system based on knowledge distillation

Abstract

The invention relates to the technical field of target detection algorithms, and discloses a target detection method and system based on knowledge distillation, and the method comprises the steps: taking a target image as a data basis, carrying out the feature extraction of the target image through a backbone network, and selecting an extracted first feature map, and performing deep feature fusion and shallow feature fusion on features extracted by the backbone network through a first RPN feature fusion model, selecting a feature value corresponding to an extracted feature map, generating a candidate frame, supervising a loss value of the candidate frame through online supervision RPN, identifying a feature model through a second target, and inputting a target image. And determining a second feature map through a second backbone network and a second RPN feature fusion module, calculating classification and regression loss, finally aligning candidate frame features through a distillation loss function, and improving the reliability of feature migration and enhancing the generalization effect of the model through a similar distillation loss function.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection algorithms and discloses a target detection method and system based on knowledge distillation. Background Art

[0002] A target detection method based on knowledge distillation is one of the key technologies in modern manufacturing. It combines a variety of advanced technologies to achieve precise control and automation. Electronic control technology is the core of this system. It covers digital control, PLC, sensor technology, etc., which are used to monitor and control the movement, position and speed of equipment in real time. These technologies enable the control system to achieve high-precision motion control and synchronous operation. However, many knowledge distillation methods need to be trained on large-scale data sets, which leads to a large consumption of computing resources and limits the popularity and application of these methods. Secondly, in some cases, the distillation process may lead to a decline in performance. Especially in complex tasks such as target detection, it is a difficult problem to maintain the performance advantage of the distillation model. Distillation usually requires the use of soft labels generated by the teacher model. These soft labels may sometimes deviate from the real targets of the target detection task, affecting the distillation effect. Finally, some knowledge distillation methods perform well on specific data sets, but their generalization ability on unseen data is insufficient, which limits their reliability and stability in practical applications. At this time, a new knowledge distillation method for target recognition is urgently needed.

[0003] For example, the patent with publication number CN116205290A discloses a knowledge distillation method and device based on intermediate feature knowledge fusion, which feeds image data into a teacher model and a second target recognition feature model, and extracts intermediate features of each stage of the teacher model and the second target recognition feature model; constructs a stage-level residual connection, and realizes feature knowledge fusion of the intermediate features of a certain stage of the second target recognition feature model with the features of the previous stage; the teacher model and the fused second target recognition feature model are subjected to global average pooling respectively to construct a semantic category feature vector, and the cross entropy loss is calculated for the feature vector to maximize feature similarity; the classification loss of the predicted output of the second target recognition feature model and the category label are weighted summed with the feature vector similarity loss to train the second target recognition feature model. It also includes a knowledge distillation system based on intermediate feature knowledge fusion. Compared with the prior art, the present invention fully integrates the intermediate feature knowledge of the teacher model, and the knowledge distillation performance is better;

[0004] For example, the patent with publication number CN115861997A discloses a license plate detection and recognition method guided by knowledge distillation of key foreground features. The spatial attention map of the backbone network features is learned, and each element in the attention map represents the probability of being the importance of the foreground. The spatial attention map is then learned simultaneously in the backbone network of the teacher model and the second target recognition feature model, and the spatial attention map of the second target recognition feature model is encouraged to imitate the spatial attention map of the teacher model. The proposed method emphasizes the learning of foreground objects by the second target recognition feature model, and suppresses the learning of background pixels by the second target recognition feature model, so that the second target recognition feature model knows which pixel information should be learned, thereby improving the efficiency and accuracy of model learning. It can not only be used for license plate detection and recognition tasks, but also for a variety of target detection tasks with large-scale data.

[0005] The above-mentioned patent only relies on the backbone network spatial attention map for screening foreground and background, which is too coarse. There is not enough prior knowledge to guide the generation of detailed foreground and background knowledge. The student and teacher networks are only aligned on the feature map and cannot pay attention to the area of ​​interest. Although the use of average pooling can simplify the feature map, it will also lead to the loss of some information, especially the weakening of the ability to capture fine-grained features. The extraction process of intermediate features is too rough, and the loss of semantic features in the extracted image is particularly serious. Summary of the invention

[0006] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0007] In order to solve the above technical problems, the main purpose of the present invention is to provide a target detection method based on knowledge distillation, comprising:

[0008] S1. Select the first target recognition feature model and freeze the model parameters of the backbone network and the FPN feature fusion module;

[0009] S2, using the target image as the data basis, extracting features of the target image through the backbone network, and selecting the extracted first feature map;

[0010] S3, perform deep feature fusion and shallow feature fusion on the features extracted by the backbone network through the first RPN feature fusion model, select the feature values ​​corresponding to the extracted feature map, generate candidate boxes, and supervise the loss value of the candidate boxes through online supervision RPN;

[0011] S4, select the second target recognition feature model and input the target image, determine the second feature map through the second backbone network and the second RPN feature fusion module, input the second feature map into the fully connected layer and the classification network, and calculate the classification and regression losses;

[0012] S5, the feature value is input into the second RPN network to generate potential candidate boxes;

[0013] S6, receiving the first feature map through the first RolAlign, receiving the size of the second feature map through the second RolAlign, unifying the first feature map and the second feature map through weight sharing, and generating a candidate box from the first feature map and the second feature map after the unified size;

[0014] S7, aligning the features of the first feature map and the second feature map with the candidate box through a feature similarity distillation loss function;

[0015] S8. Complete the object detection of the image by distilling the aligned first feature map and the second feature map and the features of the candidate box.

[0016] As a preferred solution of a target detection method based on knowledge distillation of the present invention, wherein:

[0017] The target image is input into n networks for mapping through activation functions, and the target image is mapped out through the i-th convolutional layer;

[0018] The target image is feature extracted through the n networks, and the feature maps of the features of the last three stages are selected. The target image result features are output to generate the first candidate frame through the fusion of the first feature fusion model and the shallow features. The generation of the first candidate frame is supervised by the online supervision RPN.

[0019] The online supervision RPN is used to supervise the generation of candidate boxes. The candidate box loss is supervised by the cross entropy loss of binary classification and position regression. The loss supervision function calculation expression is as follows:

[0020]

[0021] Where x is y i represents the label of the foreground of the i-th sample, p i Represents the predicted probability of the i-th sample prospect. i Indicates the position of the true candidate box, Indicates the location of the predicted candidate box.

[0022] As a preferred solution of a target detection method based on knowledge distillation of the present invention, wherein:

[0023] The second feature map of fixed size obtained by the second ROLAlign is sent to the fully connected layer and the classification layer, and the final precise position of the target is predicted by bounding box regression. The classification and regression losses are calculated as follows:

[0024]

[0025] where y i j represents the label of the jth category of the ith sample, p i j represents the predicted probability of the jth category of the ith sample. i Indicates the position of the true candidate box, Indicates the position of the predicted candidate box, L Aux_reg The function that measures the difference between the predicted value and the true value of the bounding box, L Aux_cls is the classification cross entropy loss function, smooth L1 (x) is a function that measures the difference between the predicted value and the true value of the box.

[0026] As a preferred solution of a target detection method based on knowledge distillation of the present invention, wherein:

[0027] The features extracted by the first ROLAlign are recorded as The features proposed by the second ROLAlign module are denoted as Where b represents the batch of target images, c represents the number of foreground boxes, h represents the height of the feature map, and w represents the width of the feature map.

[0028] As a preferred solution of a target detection method based on knowledge distillation of the present invention, wherein:

[0029] The feature R T The tensor of shape b×c×h×w is rearranged into b*c h*w feature vectors and denoted as F T ={T 1 ,T 2 ,...,T c},R S The eigenvector after permutation is denoted as F S ={S 1 ,S 2 ,...,S c}, through the feature R T With feature R S The cosine similarity loss term is calculated and the calculation expression is as follows:

[0030]

[0031] Among them, L sim The characteristic R T With feature RS The loss value of similarity, N is the number of target instances, and i is an ordinal number taking the values ​​1, 2, 3, ..., N.

[0032] A target detection system based on knowledge distillation, comprising:

[0033] A feature module, including a first target recognition feature model, a second target recognition feature model, a first feature fusion model, and a second feature fusion model, and an online supervision RPN;

[0034] An error module includes a first ROLAlign and a second ROLAlign;

[0035] Knowledge distillation module, used for knowledge distillation loss detection.

[0036] As a preferred solution of the target detection system based on knowledge distillation of the present invention, wherein:

[0037] Processing output features through the first target recognition feature model feature map;

[0038] Processing the output features through the second target recognition feature model feature map;

[0039] Select the output features of the first target recognition feature model after three processings and sort the output features, and input the sorted features into the first feature fusion model;

[0040] The third features after the output of the second target recognition feature model are selected and the output features are sorted into features, and the sorted features are input into the second feature fusion model.

[0041] As a preferred solution of the target detection system based on knowledge distillation of the present invention, wherein:

[0042] The fused features of the target image are output through one end of the first feature fusion model, and the first RPN feature fusion model integrates the fused features of the target image to perform binary classification, and the feature values ​​are fitted through the regression algorithm. The other end of the first feature fusion model outputs to the first ROLAlign;

[0043] The fused features of the target image are outputted through the second feature fusion, and the detection head integrates and classifies the fused features of the target image, and fits the feature values ​​through a regression algorithm.

[0044] Beneficial effects of the present invention:

[0045] The present invention improves the reliability of feature migration and the generalization effect of the model by referencing the online similarity distillation loss function. Secondly, the RPN in the two-stage detection is used to generate high-quality candidate boxes, and each candidate box area is aligned to a feature map of a fixed size through ROLAlign. The strategy of feature similarity distillation is used to continuously narrow the gap between the teacher model and the student model in the candidate area features, thereby improving the accuracy of the single-stage detector. Finally, this application combines the advantages of high accuracy of the two-stage detector and fast reasoning of the single-stage detector, which not only ensures that the reasoning speed of the model meets the real-time requirements, but also greatly improves the accuracy of the single-stage model. In the entire model reasoning stage, the network part only includes the backbone network of the student model, the FPN feature fusion module and the detection head, without adding any network modules, ensuring that the reasoning speed of the single-stage detector is not affected.

[0046] The architecture design adopted in this application has low restrictions on the selection of the first target recognition feature model and supports the distillation of homogeneous or heterogeneous models. Regardless of the specific architecture of the teacher detector, as long as it performs well, it can be used to distill other single-stage detectors. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0048] Figure 1 A flow chart of a target detection method based on knowledge distillation of the present invention;

[0049] Figure 2 This is a composition diagram of a target detection system based on knowledge distillation according to the present invention;

[0050] Figure 3 This is a topological diagram of a target detection system based on knowledge distillation of the present invention. DETAILED DESCRIPTION

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0052] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0053] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0054] Embodiment 1:

[0055] like Figure 1 As shown, a target detection method based on knowledge distillation includes:

[0056] S1. Select the first target recognition feature model and freeze the model parameters of the backbone network and the FPN feature fusion module;

[0057] Among them, through n backbone networks, one backbone network contains i convolutional layers to form a first target recognition feature model;

[0058] The backbone network and FPN feature fusion module play a key role in target detection, responsible for extracting features from the original image and forming multi-scale feature maps. In order to maintain the complexity of the teacher model but at the same time reduce the computational burden of the second target recognition feature model, the parameters of these models can be frozen.

[0059] The first target recognition feature model extracts target image features, where the target image features include other image information and the target image;

[0060] By freezing the weights of the convolutional layers of the backbone network and the FPN feature fusion module, high-quality feature representation is retained while reducing the complexity of learning the second target recognition feature model;

[0061] The model parameters are learnable parameters in the neural network;

[0062] The weight sharing is to use the same weight parameters at different positions and levels of the neural network; through weight sharing, each neuron only focuses on a small area of ​​the input data, and this small area is called the local receptive field.

[0063] S2, using the target image as the data basis, extracting features of the target image through the backbone network, and selecting the extracted first feature map;

[0064] Among them, the target image is input into n networks and mapped through the activation function. The calculation expression of the target image mapped by the i-th convolutional layer is as follows:

[0065] Y i =F i (X i )

[0066] Among them, Y i Output tensor for the target image, X i is the input target image, F i () is the activation function.

[0067] S3, perform deep feature fusion and shallow feature fusion on the features extracted by the backbone network through the first RPN feature fusion model, select the feature value corresponding to the extracted feature map, generate the first candidate box, and supervise the loss value of the candidate box through the online supervision RPN;

[0068] Deep feature fusion is used to extract features in the deeper part of the network and identify object categories;

[0069] Furthermore, deep feature fusion has stronger semantic information;

[0070] Shallow features are used to extract features from the shallower parts of the network. These features retain more spatial information and edge details, which helps to locate the position of objects more accurately.

[0071] The target image is feature extracted through n networks, and the feature maps of the last three stages are selected. The target image result features are output through the fusion of the first feature fusion model and the shallow features to generate the first candidate box. The generation of the first candidate box is supervised by the online supervision RPN. The candidate box contains rich foreground and background knowledge.

[0072] For example, the target image is used as input, and it will go through the extraction of backbone network features in turn, and select the feature maps of the last three stages, P1, P2, and P3. These feature maps are fused by the deep and shallow features of the feature fusion module, and the obtained F1, F2, and F3 features continue to pass through an RPN network to generate potential candidate boxes, which contain rich foreground and background knowledge;

[0073] The online supervision RPN is used to supervise the generation of candidate boxes. The candidate box loss is supervised by the cross entropy loss of binary classification and position regression. The loss supervision function calculation expression is as follows:

[0074]

[0075] Where x is y i represents the label of the foreground of the i-th sample, p i Represents the predicted probability of the i-th sample prospect. i Indicates the position of the true candidate box, Indicates the location of the predicted candidate box.

[0076] S4, select the second target recognition feature model and input the target image, determine the second feature map through the second backbone network and the second RPN feature fusion module, input the second feature map into the fully connected layer and the classification network, and calculate the classification and regression losses;

[0077] For the second target recognition feature model, the same target image is input and passes through the lightweight backbone network and the lightweight FPN feature fusion module in sequence to obtain the feature maps of F1', F2', and F3'. According to the experience of the teacher model, the selected candidate boxes contain rich regions of interest, which can better guide the student model to focus on the characteristics of the target.

[0078] Furthermore, the teacher model is a large-scale or high-performance model that has been fully trained. It has higher accuracy and stronger generalization ability, and the teacher model has more layers and greater width.

[0079] During the knowledge distillation process, the teacher model is used to provide information such as “soft targets” or intermediate-level features, which are richer than hard labels and reflect the similarity relationships between categories.

[0080] Furthermore, the student model is a relatively small and simple model whose purpose is to learn and imitate the behavior of the teacher model.

[0081] Furthermore, the student model adjusts its own weights to optimize its performance by receiving information from the teacher model (such as output probability distribution, feature maps, etc.), thereby achieving better detection accuracy while maintaining low computational overhead.

[0082] The second feature map of fixed size obtained from the second ROLAlign is sent to the subsequent fully connected layer and classification network to perform multi-classification tasks of foreground and background. At the same time, the bounding box regression is used to predict the final precise position of the target. The classification and regression losses are calculated as follows:

[0083]

[0084] where y i j represents the label of the jth category of the ith sample, p i j represents the predicted probability of the jth category of the ith sample. i Indicates the position of the true candidate box, Indicates the position of the predicted candidate box, L Aux_reg The function that measures the difference between the predicted value and the true value of the bounding box, L Aux_cls is the classification cross entropy loss function, smooth L1 (x) is a function that measures the difference between the predicted value and the true value of the box;

[0085] The RPN in the two-stage detection generates high-quality candidate boxes, and each candidate box area is aligned to a feature map of a fixed size through ROLAlign. The strategy of feature similarity distillation is used to continuously narrow the gap between the teacher model and the student model in the candidate region features, thereby improving the accuracy of the single-stage detector.

[0086] S5, the feature value is input into the second RPN network to generate potential candidate boxes;

[0087] After being processed by the second RPN network, each candidate box will get a candidate box offset and objectness score. This information can be used to correct each candidate box through the candidate box offset and objectness score output by the second RPN network to obtain a more accurate candidate box position, and then the candidate box is screened and adjusted to become a potential candidate box.

[0088] S6, receiving the first feature map through the first RolAlign, receiving the size of the second feature map through the second RolAlign, unifying the first feature map and the second feature map through weight sharing, and generating a candidate box from the first feature map and the second feature map after the unified size;

[0089] The first ROlAlign and the second ROlAlign are essentially the same algorithm model. Since the first ROlAlign is for the first target recognition feature model and the second ROlAlign is for the second target recognition feature model, they are distinguished in writing.

[0090] The first target recognition feature model and the second target recognition feature model use candidate boxes as guidance, and use ROLAlign to unify candidate boxes of different sizes into a feature map of a fixed size. These feature maps are the results of each generated candidate box corresponding to the feature image, and the obtained small-size feature map is more accurate and complete information;

[0091] By introducing an additional online supervision mechanism, ROLAlign can generate high-quality, fine and smooth feature maps. Finally, the proposed feature similarity distillation loss function is used to align the features between the two, which greatly improves the reliability of feature migration and enhances the generalization effect of the model.

[0092] The first feature extracted by ROLAlign is recorded as The features proposed by the second ROLAlign module are denoted as Where b represents the batch of target images, c represents the number of foreground boxes, h represents the height of the feature map, and w represents the width of the feature map;

[0093] The feature R T The tensor of shape b×c×h×w is rearranged into b*c h*w feature vectors and denoted as FT ={T 1 ,T 2 ,...,T c},R S The eigenvector after permutation is denoted as F S ={S 1 ,S 2 ,...,S c}, through the feature R T With feature R S The cosine similarity loss term is calculated and the calculation expression is as follows:

[0094]

[0095] Among them, L sim The characteristic R T With feature R S The loss value of similarity, N is the number of target instances, and i is an ordinal number taking the values ​​1, 2, 3, ..., N.

[0096] S7. Align the first feature map and the second feature map with the features of the candidate box through the feature similarity distillation loss function to calculate the loss.

[0097] The final loss function uses the following formula to calculate the total distillation loss:

[0098] L total =L stu_task +λ 1 L rpn_cls +λ 2 L rpn_reg +λ 3 L Aux_cls +λ 4 L Aux_reg +λ 5 L sim

[0099] where λ 1 ,λ 2 ,λ 3 ,λ 4 ,λ 5 It is variable and needs to be manually adjusted according to the specific teacher model and the second target recognition feature model. stu_task represents the loss function related to the detection task of the student model;

[0100] The proposed feature similarity distillation loss function aligns the features between the two, greatly improving the reliability of feature migration and enhancing the generalization effect of the model;

[0101] S8, completing object detection of the image by distilling the aligned first feature map and the second feature map and the features of the candidate box;

[0102] As an optional implementation, the first feature map is used to generate a first candidate box, and the second feature map is used to generate a second candidate box, wherein the first candidate box and the second candidate box both contain integrated image feature information, and the information of the first feature map and the second feature map are integrated together by distillation alignment and associated with the candidate box, thereby realizing object detection;

[0103] The first feature map and the second feature map are aligned by distillation through upsampling or downsampling operations to ensure that they can correspond to each other in space, and then feature fusion is performed to fuse the aligned first feature map and the second feature map together. The methods that can be used include but are not limited to element-level addition and concatenation. The purpose is to allow the model to utilize feature information at different levels.

[0104] Furthermore, a region proposal network (RPN) or other mechanism is used to generate candidate boxes containing potential targets. For each candidate box, operations such as RoI Align are performed on the fused feature map to extract the feature vector corresponding to the candidate box. The extracted feature vector is used for classification prediction, and then the category of the object in the candidate box and bounding box regression are determined. Finally, the overlapping detection results are filtered to retain the most likely detection box.

[0105] Bounding box regression is used to adjust the position of the candidate box so that it surrounds the object more accurately.

[0106] Embodiment 2:

[0107] like Figure 2 As shown, a target detection system based on knowledge distillation includes:

[0108] A data module, including a target image to be processed;

[0109] A feature module, including a first target recognition feature model, a second target recognition feature model, a first feature fusion model, and a second feature fusion model, and an online supervision RPN;

[0110] An error module includes a first ROLAlign and a second ROLAlign;

[0111] Knowledge distillation module, used for knowledge distillation loss detection.

[0112] like Figure 3 As shown, the target image is input into the first target recognition feature model and the second target recognition feature model respectively;

[0113] The output features P1, P2, P3, P4, and P5 are processed sequentially through the first target recognition feature model feature map, and the output features F1, F2, F3, F4, and F5 are processed sequentially through the second target recognition feature model feature map;

[0114] Select the output features P3, P4, and P5 of the first target recognition feature model after three processings, organize the features into F1, F2, and F3, and input them into the first feature fusion model; select the output features P3', P4', and P5' of the second target recognition feature model after three processings, organize the features into F1', F2', and F3', and input them into the second feature fusion model;

[0115] Furthermore, the fused features of the target image are outputted through one end of the first feature fusion model, and the fused features of the target image are classified into two categories by RPN integration, and the feature values ​​are fitted through the regression algorithm. The output of the other end of the first feature fusion model enters the second ROLAlign;

[0116] Combining the advantages of high accuracy of the two-stage detector and fast reasoning of the single-stage detector, not only does it ensure that the reasoning speed of the model meets the real-time requirements, but it also greatly improves the accuracy of the single-stage model. During the entire model reasoning stage, the network part only includes the backbone network of the student model, the FPN feature fusion module, and the detection head, without adding any network modules, ensuring that the reasoning speed of the single-stage detector is not affected.

[0117] Furthermore, the fused features of the target image are outputted through the second feature fusion, and the detection head integrates and classifies the fused features of the target image, and fits the feature values ​​through the regression algorithm;

[0118] The first ROLAlign and the second ROLAlign share weights, and the new weights and target image features are input into the fully connected layer for classification and fitting feature regression.

[0119] Importantly, it should be noted that the construction and arrangement of the present application shown in a plurality of different exemplary embodiments are only exemplary. Although only two embodiments are described in detail in this disclosure, it should be readily understood by those who refer to this disclosure that many modifications are possible, for example, the size, scale, structure, shape and ratio of various elements, and parameter values ​​(e.g., temperature, pressure, etc.), installation arrangement, use of materials, color, directional changes, etc., without substantially departing from the novel teachings and advantages of the subject matter described in the application. For example, the element shown as integrally formed can be composed of multiple parts or elements, the position of the element can be inverted or otherwise changed, and the nature or number or position of the discrete element can be changed or changed. Therefore, all such modifications are intended to be included in the scope of the present invention. The order or sequence of any process or method step can be changed or reordered according to an alternative embodiment. Any "device plus function" clause is intended to cover the structure of the execution function described in this article, and is not only structurally equivalent but also equivalent structure. Without departing from the scope of the present invention, other replacements, modifications, changes and omissions can be made in the design, operating conditions and arrangement of the exemplary embodiment. Therefore, the invention is not limited to a specific embodiment, but extends to numerous modifications still falling within the scope of the appended claims.

[0120] Additionally, in order to provide a concise description of exemplary embodiments, all features of an actual embodiment (ie, those features that are not relevant to the best mode presently contemplated for carrying out the invention or those that are not relevant to implementing the invention) may not be described.

[0121] It should be understood that in the development of any actual implementation, as in any engineering or design project, numerous implementation-specific decisions may be made. Such a development effort may be complex and time-consuming, but for those of ordinary skill having the benefit of this disclosure, the development effort will be a routine task of design, fabrication, and production without undue experimentation.

[0122] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A target detection method based on knowledge distillation, comprising: S1, select the first target recognition feature model, and freeze the model parameters of the backbone network and FPN feature fusion module; S2, using the target image as the data basis, extracting features of the target image through the backbone network, and selecting the extracted first feature map; S3, perform deep feature fusion and shallow feature fusion on the features extracted by the backbone network through the first RPN feature fusion model, select the feature values ​​corresponding to the extracted feature map, generate candidate boxes, and supervise the loss value of the candidate boxes through online supervision RPN; S4, select the second target recognition feature model and input the target image, determine the second feature map through the second backbone network and the second RPN feature fusion module, input the second feature map into the fully connected layer and the classification network, and calculate the classification and regression losses; S5, the feature value is input into the second RPN network to generate potential candidate boxes; S6, receiving the first feature map through the first RolAlign, receiving the size of the second feature map through the second RolAlign, unifying the first feature map and the second feature map through weight sharing, and generating a candidate box from the first feature map and the second feature map after the unified size; S7, aligning the features of the first feature map and the second feature map with the candidate box through a feature similarity distillation loss function; S8. Complete the object detection of the image by distilling the aligned first feature map and the second feature map and the features of the candidate box.

2. The target detection method based on knowledge distillation according to claim 1, characterized in that: The target image is input into n networks and mapped through the activation function, and the target image is mapped out through the i-th convolutional layer; The target image is feature extracted through the n networks, and the feature maps of the features of the last three stages are selected. The target image result features are output to generate the first candidate frame through the fusion of the first feature fusion model and the shallow features. The generation of the first candidate frame is supervised by the online supervision RPN. The online supervision RPN is used to supervise the generation of candidate boxes. The candidate box loss is supervised by the cross entropy loss of binary classification and position regression. The loss supervision function calculation expression is as follows: Where x is y i represents the label of the foreground of the i-th sample, p i represents the predicted probability of the i-th sample prospect, t i Indicates the position of the true candidate box, Indicates the location of the predicted candidate box.

3. The target detection method based on knowledge distillation according to claim 2, characterized in that: The second feature map of fixed size obtained by the second ROLAlign is fed into the fully connected layer and the classification layer, which predicts the final precise position of the target through bounding box regression, and calculates the classification and regression losses as follows: where y i j represents the label of the jth category of the ith sample, p i j represents the predicted probability of the jth category of the i-th sample, t i Indicates the position of the true candidate box, Indicates the position of the predicted candidate box, L Aux_reg The function that measures the difference between the predicted value and the true value of the bounding box, L Aux_cls is the classification cross entropy loss function, smooth L1 (x) is a function that measures the difference between the predicted value and the true value of the box.

4. The target detection method based on knowledge distillation according to claim 3, characterized in that: The features extracted by the first ROLAlign are recorded as The features proposed by the second ROLAlign module are denoted as Where b represents the batch of target images, c represents the number of foreground boxes, h represents the height of the feature map, and w represents the width of the feature map.

5. The target detection method based on knowledge distillation according to claim 4, characterized in that: The feature R T The tensor of shape b×c×h×w is rearranged into b*c h*w feature vectors and denoted as F T ={T1,T2,...,T c },R S The eigenvector after permutation is denoted as F S ={S1,S2,...,S c }, through the feature R T With feature R S The cosine similarity loss term is calculated and the calculation expression is as follows: Among them, L sim The characteristic R T With feature R S The loss value of similarity, N is the number of target instances, and i is an ordinal number taking the value 1, 2, 3, ... N.

6. A target detection system based on knowledge distillation, used to implement a target detection method based on knowledge distillation according to any one of claims 1 to 5, characterized in that: include, A feature module, including a first target recognition feature model, a second target recognition feature model, a first feature fusion model, and a second feature fusion model, and an online supervision RPN; An error module includes a first ROLAlign and a second ROLAlign; Knowledge distillation module, used for knowledge distillation loss detection.

7. The object detection system based on knowledge distillation according to claim 6, characterized in that: Processing output features through the first target recognition feature model feature map; Processing the output features through the second target recognition feature model feature map; Select the output features of the first target recognition feature model after three processings and sort the output features, and input the sorted features into the first feature fusion model; The third features after the output of the second target recognition feature model are selected and the output features are sorted into features, and the sorted features are input into the second feature fusion model.

8. The object detection system based on knowledge distillation according to claim 7, characterized in that: The fused features of the target image are output through one end of the first feature fusion model, and the first RPN feature fusion model integrates the fused features of the target image to perform binary classification, and the feature values ​​are fitted through the regression algorithm. The other end of the first feature fusion model outputs to the first ROLAlign; The fused features of the target image are outputted through the second feature fusion, and the detection head integrates and classifies the fused features of the target image, and fits the feature values ​​through a regression algorithm.

Images