Visual target detection algorithm based on large and small model collaboration
Through the collaborative training of large and small models for visual object detection, the problem of insufficient object detection performance on resource-constrained devices is solved, and efficient and low-consumption object detection effect is achieved.
Patent Information
- Application Number
- CN202411119847.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-22
AI Technical Summary
The existing high-performance object detection models are limited in applications on devices with limited resources, and small model training methods are difficult to achieve comparable performance to large models.
Through collaborative training of the visual object detection large model LargeYOLO and the small model EfficientYOLO, the public object detection data set COCO2017 is used to construct and optimize the training and verification process to achieve efficient training and inference of the small model EfficientYOLO.
EfficientYOLO obtains performance comparable to that of the large model LargeYOLO when the number of parameters is small, reducing the computing resource requirements and is suitable for efficient object detection in resource-constrained environments.
Smart Images

Figure CN119006791A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a visual target detection algorithm based on large and small model collaboration, which is applicable to the technical field of target detection in computer vision. Background Art
[0002] In the field of computer vision, object detection technology plays a vital role. It is widely used in many fields such as autonomous driving, security monitoring, and medical image analysis. With the development of deep learning, object detection algorithms have made significant progress. However, current high-performance object detection models usually require a lot of computing resources and storage space, which limits their application on resource-constrained devices (such as mobile devices and embedded systems).
[0003] In order to achieve efficient target detection in resource-constrained environments, researchers have proposed a variety of methods to compress models and improve computational efficiency. However, these methods usually require a trade-off between model performance and computing resources, resulting in poor performance of the model under resource-constrained conditions. Existing small model training methods mainly rely on supervised learning, but this approach makes it difficult for small models to achieve performance comparable to that of large models. Therefore, research on efficient training methods for small models has important research significance and practical application value. Summary of the invention
[0004] The present invention proposes a large-scale model collaborative visual target detection algorithm, named EfficientYOLO. Through the collaborative training of the visual target detection large model LargeYOLO and the visual target detection small model EfficientYOLO, EfficientYOLO can still achieve performance comparable to that of the large model LargeYOLO with a small number of parameters. This method can not only effectively improve the detection performance of the small model EfficientYOLO, but also reduce the computing resource requirements during model training and reasoning, thereby achieving efficient target detection in a resource-constrained environment.
[0005] In order to achieve the above object, the present invention provides the following technical solutions:
[0006] A visual target detection algorithm based on large and small model collaboration, characterized by comprising the following steps:
[0007] Step S1: Construct training set, validation set and test set based on the public object detection dataset COCO2017;
[0008] Step S2: construct a large model algorithm LargeYOLO for visual object detection with 50 million parameters based on the YOLO architecture, and use the constructed training set to perform supervised training on LargeYOLO until the model converges to the optimal performance;
[0009] Step S3: construct a small model algorithm EfficientYOLO for visual target detection with 1 million parameters based on the YOLO architecture, and use the constructed training set and the converged large model algorithm LargeYOLO for visual target detection to perform collaborative supervision training on EfficientYOLO until EfficientYOLO converges to the optimal performance;
[0010] Step S4: Test the converged EfficientYOLO on the constructed test set, and evaluate the performance of the model EfficientYOLO based on the detection results.
[0011] Furthermore, the step S1 specifically includes:
[0012] Step S101: Obtain the public object detection dataset COCO2017.
[0013] Step S102: Divide the COCO2017 dataset into a training set, a validation set, and a test set. The training set contains 118,287 image label pairs, the validation set contains 5,000 image label pairs, and the test set contains 40,670 image label pairs.
[0014] Furthermore, the step S2 specifically includes:
[0015] Step S201: Preprocess the training data. First, use bilinear interpolation to resize the original image to , and adjust the corresponding true label, then enhance the image data, such as random scaling and random rotation, and finally normalize the image and input it into the LargeYOLO model for training;
[0016] Step S202: The feature extraction module of the LargeYOLO model extracts features from the input image, and then inputs the extracted multi-layer features into the detection head module for target detection training;
[0017] Step S203: In the training phase of the LargeYOLO model, the confidence loss function, classification loss function and positioning loss function are used to supervise the target results obtained by the model detection. The overall loss function can be used To express it, the calculation formula is:
[0018] (1)
[0019] in, Represents the confidence loss of the detected target; Represents the classification loss of the detection target; Represents the positioning loss of the detected target; , and is the weight coefficient;
[0020] , and The calculation formula is as follows:
[0021] (2)
[0022] (3)
[0023] in represents the Intersection of Union (IoU) between the predicted target bounding box and the true target bounding box, is the predicted value, is the prediction confidence obtained by the Sigmoind function, is the number of positive and negative samples;
[0024] The calculation formula is as follows:
[0025] (4)
[0026] (5)
[0027] in Represents the predicted target bounding box Is there a Class target, is the predicted value, for The target probability obtained by the Sigmoid function, is the number of positive samples.
[0028] The calculation formula is as follows:
[0029] (6)
[0030] (7)
[0031] (8)
[0032] (9)
[0033] (10)
[0034] in , , and For the The regression parameters of the predicted target; , , and is the annotation information of the real label, that is, the coordinates of the center point , and width and height;
[0035] Step S204: During the training process, the Adam optimization algorithm is used to iteratively update the LargeYOLO network parameters, and the learning rate is decayed by the polynomial method. During the training process, the LargeYOLO model performance is verified using the validation set every 200 iterations to ensure that the LargeYOLO model converges to the optimal performance;
[0036] Step S205: Use the converged LargeYOLO network model to perform forward operations on the test set images. , get the test results , the corresponding real label information is , calculate the test indicators classification accuracy (Precision, P), recall (Recall, R) and average accuracy (mean Average Precision, mAP), you can get the test performance of LargeYOLO, while the model parameters (Params) and average detection time (Mean Time to Detect, MT) are also one of the test indicators. The calculation formulas for Precision and Recall are as follows:
[0037] (11)
[0038] (12)
[0039] in (True Position, TP) is the number of positive samples correctly predicted, (False Position, FP) indicates the number of negative samples that are incorrectly predicted. (False Negative, FN) indicates the number of positive samples that are incorrectly predicted.
[0040] Furthermore, the step S3 specifically includes:
[0041] Step S301: Preprocess the training data. First, use bilinear interpolation to resize the original image to , and adjust the corresponding true label, then enhance the image data, such as random scaling and random rotation, and finally normalize the image and input it into the EfficientYOLO model for training;
[0042] Step S302: The feature extraction module of the EfficientYOLO model extracts features from the input image, and then inputs the extracted multi-layer features into the detection head module for target detection training;
[0043] Step S303: In the training phase of the EfficientYOLO model, the confidence loss function, classification loss function, positioning loss function and interaction loss function are used to supervise the target results obtained by the model detection. The overall loss function can be used To express it, the calculation formula is:
[0044] (13)
[0045] in, Represents the confidence loss of the detected target; Represents the classification loss of the detection target; Represents the positioning loss of the detected target; Represents the interactive loss function between the large model LargeYOLO and the small model EfficientYOLO. , , and is the weight coefficient.
[0046] , and The calculation formula is as follows:
[0047] (14)
[0048] (15)
[0049] in represents the Intersection of Union (IoU) between the predicted target bounding box and the true target bounding box, is the predicted value, is the prediction confidence obtained by the Sigmoind function, is the number of positive and negative samples;
[0050] The calculation formula is as follows:
[0051] (16)
[0052] (17)
[0053] in Represents the predicted target bounding box Is there a Class target, is the predicted value, for The target probability obtained by the Sigmoid function, is the number of positive samples;
[0054] The calculation formula is as follows:
[0055] (18)
[0056] (19)
[0057] (20)
[0058] (twenty one)
[0059] (twenty two)
[0060] in , , and For the The regression parameters of the predicted target; , , and is the annotation information of the real label, that is, the coordinates of the center point , and width and height;
[0061] The calculation formula is as follows:
[0062] (twenty three)
[0063] in For LargeYOLO The feature map of the layer, For EfficientYOLO The feature map of the layer, It is the feature layer of LargeYOLO and EfficientYOLO.
[0064] Step S304: During the training process, the Adam optimization algorithm is used to iteratively update the EfficientYOLO network parameters, and the learning rate is decayed by the polynomial method. During the training process, the performance of the EfficientYOLO model is verified using the validation set every 200 iterations to ensure that the EfficientYOLO model converges to the optimal performance;
[0065] Furthermore, the step S4 specifically includes:
[0066] Step S401: Use the converged EfficientYOLO network model to perform forward operations on the test set images. , get the test results , the corresponding real label information is , calculate the test indicators classification accuracy (Precision, P), recall (Recall, R) and average accuracy (mean Average Precision, mAP), you can get the test performance of LargeYOLO, while the model parameters (Params) and average detection time (Mean Time to Detect, MT) are also one of the test indicators. The calculation formulas for Precision and Recall are as follows:
[0067] (twenty four)
[0068] (25)
[0069] in (True Position, TP) is the number of positive samples correctly predicted, (False Position, FP) indicates the number of negative samples that are incorrectly predicted. (False Negative, FN) indicates the number of positive samples that are incorrectly predicted.
[0070] The beneficial effect of the present invention is that the large and small model collaborative visual target detection algorithm EfficientYOLO proposed by the present invention is an efficient visual target detection algorithm. EfficientYOLO has the characteristics of small number of parameters and high accuracy, making it more suitable for application on resource-constrained devices (such as mobile devices and embedded systems). BRIEF DESCRIPTION OF THE DRAWINGS
[0071] Figure 1This is a flow chart of the EfficientYOLO visual target detection algorithm based on large and small model collaboration in Example 1;
[0072] Figure 2 It is the structure diagram of the LargeYOLO and EfficientYOLO network models in Example 1;
[0073] Figure 3 It is a diagram of the structural parameters of the LargeYOLO and EfficientYOLO network models in Example 1;
[0074] Figure 4 This is a comparison of the EfficientYOLO method of the present invention in Example 1 on the test set evaluation indicators. DETAILED DESCRIPTION
[0075] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0076] Example 1
[0077] See also Figure 1-Figure 3 ,This implementation provides a visual target detection algorithm based on ,large and small model collaboration.
[0078] Specifically, see Figure 1 , this method specifically includes:
[0079] Step S1: Obtain the public object detection and object tracking dataset (COCO2017) and construct the training set, validation set, and test set.
[0080] More specifically, the constructed training set contains 118,287 image-label pairs, the validation set contains 5,000 image-label pairs, and the test set contains 40,670 image-label pairs.
[0081] Step S2: Construct a large model LargeYOLO and a small model EfficientYOLO based on deep neural network for visual target detection.
[0082] More specifically, the specific structures of the constructed LargeYOLO and EfficientYOLO network models are as follows Figure 2 The detailed parameter settings are as shown in Figure 3 shown.
[0083] Step S3: On the Pytorch deep learning framework, supervise the LargeYOLO network model using the constructed training set, and verify the model performance using the validation set every certain number of iterations until the model converges to the optimal performance;
[0084] More specifically, the LargeYOLO training process includes:
[0085] Step S301: Preprocess the training data. First, use bilinear interpolation to resize the original image to , and adjust the corresponding true label, then enhance the image data, such as random scaling and random rotation, and finally normalize the image and input it into the LargeYOLO model for training;
[0086] Step S302: The feature extraction module of the LargeYOLO model extracts features from the input image, and then inputs the extracted multi-layer features into the detection head module for target detection training;
[0087] Step S303: In the training phase of the LargeYOLO model, the confidence loss function, classification loss function and positioning loss function are used to supervise the target results obtained by the model detection. The overall loss function can be used To express it, the calculation formula is:
[0088] (1)
[0089] in, Represents the confidence loss of the detected target; Represents the classification loss of the detection target; Represents the positioning loss of the detected target; , and is the weight coefficient;
[0090] , and The calculation formula is as follows:
[0091] (2)
[0092] (3)
[0093] in represents the Intersection of Union (IoU) between the predicted target bounding box and the true target bounding box, is the predicted value, is the prediction confidence obtained by the Sigmoind function, is the number of positive and negative samples;
[0094] The calculation formula is as follows:
[0095] (4)
[0096] (5)
[0097] in Represents the predicted target bounding box Is there a Class target, is the predicted value, for The target probability obtained by the Sigmoid function, is the number of positive samples;
[0098] The calculation formula is as follows:
[0099] (6)
[0100] (7)
[0101] (8)
[0102] (9)
[0103] (10)
[0104] in , , and For the The regression parameters of the predicted target; , , and is the annotation information of the real label, that is, the coordinates of the center point , and width and height;
[0105] Step S304: During the training process, the Adam optimization algorithm is used to iteratively update the LargeYOLO network parameters, and the learning rate is decayed by the polynomial method. During the training process, the LargeYOLO model performance is verified using the validation set every 200 iterations to ensure that the LargeYOLO model converges to the optimal performance;
[0106] Step S305: Use the converged LargeYOLO network model to perform forward operations on the test set images. , get the test results , the corresponding real label information is , calculate the test indicators classification accuracy (Precision, P), recall (Recall, R) and average accuracy (mean Average Precision, mAP), you can get the test performance of LargeYOLO, while the model parameters (Params) and average detection time (Mean Time to Detect, MT) are also one of the test indicators. The calculation formulas for Precision and Recall are as follows:
[0107] (11)
[0108] (12)
[0109] in (True Position, TP) is the number of positive samples correctly predicted, (False Position, FP) indicates the number of negative samples that are incorrectly predicted. (False Negative, FN) indicates the number of positive samples that are incorrectly predicted.
[0110] Step S4: On the Pytorch deep learning framework, combined with the converged LargeYOLO, the constructed training set is used to supervise the training of the EfficientYOLO network model, and the model performance is verified using the validation set every certain number of iterations until the model converges to the optimal performance;
[0111] More specifically, the training process of EfficientYOLO includes:
[0112] Step S401: Preprocess the training data. First, use bilinear interpolation to resize the original image to , and adjust the corresponding true label, then enhance the image data, such as random scaling and random rotation, and finally normalize the image and input it into the EfficientYOLO model for training;
[0113] Step S402: The feature extraction module of the EfficientYOLO model extracts features from the input image, and then inputs the extracted multi-layer features into the detection head module for target detection training;
[0114] Step S403: In the training phase of the EfficientYOLO model, the confidence loss function, classification loss function, positioning loss function and interaction loss function are used to supervise the target results obtained by the model detection. The overall loss function can be used To express it, the calculation formula is:
[0115] (13)
[0116] in, Represents the confidence loss of the detected target; Represents the classification loss of the detection target; Represents the positioning loss of the detected target; Represents the interactive loss function between the large model LargeYOLO and the small model EfficientYOLO. , , and is the weight coefficient;
[0117] , and The calculation formula is as follows:
[0118] (14)
[0119] (15)
[0120] in represents the Intersection of Union (IoU) between the predicted target bounding box and the true target bounding box, is the predicted value, is the prediction confidence obtained by the Sigmoind function, is the number of positive and negative samples;
[0121] The calculation formula is as follows:
[0122] (16)
[0123] (17)
[0124] in Represents the predicted target bounding box Is there a Class target, is the predicted value, for The target probability obtained by the Sigmoid function, is the number of positive samples;
[0125] The calculation formula is as follows:
[0126] (18)
[0127] (19)
[0128] (20)
[0129] (twenty one)
[0130] (twenty two)
[0131] in , , and For the The regression parameters of the predicted target; , , and is the annotation information of the real label, that is, the coordinates of the center point , and width and height;
[0132] The calculation formula is as follows:
[0133] (twenty three)
[0134] in For LargeYOLO The feature map of the layer, For EfficientYOLO The feature map of the layer, It is the feature layer of LargeYOLO and EfficientYOLO;
[0135] Step S404: During the training process, the Adam optimization algorithm is used to iteratively update the EfficientYOLO network parameters, and the learning rate is decayed by the polynomial method. During the training process, the performance of the EfficientYOLO model is verified using the validation set every 200 iterations to ensure that the EfficientYOLO model converges to the optimal performance;
[0136] Step S405: Use the converged EfficientYOLO network model to perform forward operations on the test set images. , get the test results , the corresponding real label information is , calculate the test indicators classification accuracy (Precision, P), recall (Recall, R) and average accuracy (mean Average Precision, mAP), you can get the test performance of LargeYOLO, while the model parameters (Params) and average detection time (Mean Time to Detect, MT) are also one of the test indicators. The calculation formulas for Precision and Recall are as follows:
[0137] (twenty four)
[0138] (25)
[0139] in (True Position, TP) is the number of positive samples correctly predicted, (False Position, FP) indicates the number of negative samples that are incorrectly predicted. (False Negative, FN) indicates the number of positive samples that are incorrectly predicted.
[0140] Figure 4 The object detection accuracy of the proposed method EfficientYOLO on the COCO test set is demonstrated; Figure 4 The results show that the small visual target detection model EfficientNet proposed in this paper can achieve detection performance comparable to that of the large model LargeYOLO, and is a high-precision and low-cost visual target detection algorithm.
[0141] The matters not described in detail in the present invention are all known technologies to those skilled in the art.
[0142] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.
Claims
1. A visual target detection algorithm based on large and small model collaboration, characterized in that: The method comprises the following steps: Step S1: Construct training set, validation set and test set based on the public object detection dataset COCO2017; Step S2: construct a large model algorithm LargeYOLO for visual object detection with 50 million parameters based on the YOLO architecture, and use the constructed training set to perform supervised training on LargeYOLO until the model converges to the optimal performance; Step S3: construct a small model algorithm EfficientYOLO for visual target detection with 1 million parameters based on the YOLO architecture, and use the constructed training set and the converged large model algorithm LargeYOLO for visual target detection to perform collaborative supervision training on EfficientYOLO until EfficientYOLO converges to the optimal performance; Step S4: Test the converged EfficientYOLO on the constructed test set, and evaluate the performance of the model EfficientYOLO based on the detection results.
2. The large-small model collaborative visual target detection algorithm according to claim 1, characterized in that: The step S1 specifically includes: Step S101: Obtain the public object detection dataset COCO2017; Step S102: Divide the COCO2017 dataset into a training set, a validation set, and a test set. The training set contains 118,287 image label pairs, the validation set contains 5,000 image label pairs, and the test set contains 40,670 image label pairs.
3. The large-small model collaborative visual target detection algorithm according to claim 1, characterized in that: The step S2 specifically includes: Step S201: Preprocess the training data. First, use bilinear interpolation to resize the original image to , and adjust the corresponding true label, then enhance the image data, such as random scaling and random rotation, and finally normalize the image and input it into the LargeYOLO model for training; Step S202: The feature extraction module of the LargeYOLO model extracts features from the input image, and then inputs the extracted multi-layer features into the detection head module for target detection training; Step S203: In the training phase of the LargeYOLO model, the confidence loss function, classification loss function and positioning loss function are used to supervise the target results obtained by the model detection. The overall loss function can be used To express it, the calculation formula is: (1) in, Represents the confidence loss of the detected target; Represents the classification loss of the detection target; Represents the positioning loss of the detected target; , and is the weight coefficient; , and The calculation formula is as follows: (2) (3) in represents the Intersection of Union (IoU) between the predicted target bounding box and the true target bounding box, is the predicted value, is the prediction confidence obtained by the Sigmoind function, is the number of positive and negative samples; The calculation formula is as follows: (4) (5) in Represents the predicted target bounding box Is there a Class target, is the predicted value, for The target probability obtained by the Sigmoid function, is the number of positive samples; The calculation formula is as follows: (6) (7) (8) (9) (10) in , , and For the The regression parameters of the predicted target; , , and is the annotation information of the real label, that is, the coordinates of the center point , and width and height; Step S204: During the training process, the Adam optimization algorithm is used to iteratively update the LargeYOLO network parameters, and the learning rate is decayed by the polynomial method. During the training process, the LargeYOLO model performance is verified using the validation set every 200 iterations to ensure that the LargeYOLO model converges to the optimal performance; Step S205: Use the converged LargeYOLO network model to perform forward operations on the test set images. , get the test results , the corresponding real label information is , calculate the test indicators classification accuracy (Precision, P), recall (Recall, R) and average accuracy (mean AveragePrecision, mAP), you can get the test performance of LargeYOLO, while the model parameters (Params) and average detection time (Mean Time to Detect, MT) are also one of the test indicators. The calculation formulas for Precision and Recall are as follows: (11) (12) in (True Position, TP) is the number of positive samples correctly predicted, (False Position, FP) indicates the number of negative samples that are incorrectly predicted. (False Negative, FN) indicates the number of positive samples that are incorrectly predicted.
4. The visual target detection algorithm based on large and small model collaboration according to claim 1, characterized in that: The step S3 specifically includes: Step S301: Preprocess the training data. First, use bilinear interpolation to resize the original image to , and adjust the corresponding true label, then enhance the image data, such as random scaling and random rotation, and finally normalize the image and input it into the EfficientYOLO model for training; Step S302: The feature extraction module of the EfficientYOLO model extracts features from the input image, and then inputs the extracted multi-layer features into the detection head module for target detection training; Step S303: In the training phase of the EfficientYOLO model, the confidence loss function, classification loss function, positioning loss function and interaction loss function are used to supervise the target results obtained by the model detection. The overall loss function can be used To express it, the calculation formula is: (13) in, Represents the confidence loss of the detected target; Represents the classification loss of the detection target; Represents the positioning loss of the detected target; Represents the interactive loss function between the large model LargeYOLO and the small model EfficientYOLO. , , and is the weight coefficient; , and The calculation formula is as follows: (14) (15) in represents the Intersection of Union (IoU) between the predicted target bounding box and the true target bounding box, is the predicted value, is the prediction confidence obtained by the Sigmoind function, is the number of positive and negative samples; The calculation formula is as follows: (16) (17) in Represents the predicted target bounding box Is there a Class target, is the predicted value, for The target probability obtained by the Sigmoid function, is the number of positive samples; The calculation formula is as follows: (18) (19) (20) (21) (22) in , , and For the The regression parameters of the predicted target; , , and is the annotation information of the real label, that is, the coordinates of the center point , and width and height; The calculation formula is as follows: (23) in For LargeYOLO The feature map of the layer, For EfficientYOLO The feature map of the layer, It is the feature layer of LargeYOLO and EfficientYOLO; Step S304: During the training process, the Adam optimization algorithm is used to iteratively update the EfficientYOLO network parameters, and the learning rate is decayed by the polynomial method. During the training process, the EfficientYOLO model performance is verified using the validation set every 200 iterations to ensure that the EfficientYOLO model converges to the optimal performance.
5. The visual target detection algorithm based on large and small model collaboration according to claim 1, characterized in that: The step S4 specifically includes: Step S401: Use the converged EfficientYOLO network model to perform forward operations on the test set images. , get the test results , the corresponding real label information is , calculate the test indicators classification accuracy (Precision, P), recall (Recall, R) and average accuracy (meanAverage Precision, mAP), you can get the test performance of LargeYOLO, while the model parameters (Params) and average detection time (Mean Time to Detect, MT) are also one of the test indicators. The calculation formulas for Precision and Recall are as follows: (25) (26) in (True Position, TP) is the number of positive samples correctly predicted, (False Position, FP) indicates the number of negative samples that are incorrectly predicted. (False Negative, FN) indicates the number of positive samples that are incorrectly predicted.
Citation Information
Cited By
Text classification and model training method and system based on big and small model collaborative learning
CN119293252A
Air-ground cooperative target detection and feature analysis system and method
CN120318501A
