Grounding hanging ring identification method and device based on deep learning, equipment and medium

Abstract

The invention relates to the technical field of data processing, and discloses a grounding hanging ring identification method and device based on deep learning, equipment and a medium, and the method comprises the steps: carrying out the model lightweight processing of an initial model through original training data based on deep learning, and obtaining a lightweight grounding hanging ring identification model; performing hybrid enhancement processing on the original training data to obtain enhanced training data; training the lightweight grounding hanging ring identification model by enhancing the training data to obtain a reference grounding hanging ring identification model; performing small target detection pre-training based on the reference grounding hanging ring identification model to obtain a pre-trained grounding hanging ring identification model; and carrying out transfer learning processing on the pre-trained grounding hanging ring identification model to obtain a target grounding hanging ring identification model. By implementing the method provided by the invention, a more accurate target grounding hanging ring identification model can be obtained, a more accurate identification result can be identified according to the target grounding hanging ring identification model, and more accurate grounding hanging ring identification can be realized.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a grounding loop identification method, device, equipment and medium based on deep learning. Background Art

[0002] The traditional manual pole climbing for low and medium voltage overhead lines to test electricity and install and remove ground wires is not only a heavy task, but also difficult to climb and difficult to ensure personal safety when geological disasters such as ice and floods affect the line section. At present, the convenient device for installing and removing ground wires by drones for low and medium voltage overhead lines has become a hot research topic. This research focuses on drone control technology for installing and removing ground wires. However, accurate grounding ring recognition is a necessary condition for drone installation and removal of ground wires. Therefore, how to achieve more accurate grounding ring recognition has become a problem that needs to be solved urgently. Summary of the invention

[0003] The present invention provides a grounding loop identification method, device, equipment and medium based on deep learning to solve the technical problem of how to achieve more accurate grounding loop identification.

[0004] In a first aspect, a grounding loop recognition method based on deep learning is provided, comprising:

[0005] Based on deep learning, the initial model is lightweighted through the original training data to obtain a lightweight grounding loop recognition model;

[0006] Enhanced training data is obtained by performing mixed enhancement processing on the original training data;

[0007] By enhancing the training data, the lightweight grounding ring recognition model is trained to obtain the reference grounding ring recognition model;

[0008] Small target detection pre-training is performed based on the reference grounding loop recognition model to obtain a pre-trained grounding loop recognition model;

[0009] The pre-trained grounding loop recognition model is subjected to transfer learning to obtain the target grounding loop recognition model.

[0010] In a second aspect, a grounding loop recognition device based on deep learning is provided, comprising:

[0011] The first processing module is used to perform model lightweight processing on the initial model through original training data based on deep learning to obtain a lightweight grounding loop recognition model;

[0012] The second processing module is used to obtain enhanced training data by performing mixed enhancement processing on the original training data;

[0013] The third processing module is used to train the lightweight grounding ring recognition model by enhancing the training data to obtain a reference grounding ring recognition model;

[0014] A fourth processing module is used to perform small target detection pre-training based on a reference grounding loop recognition model to obtain a pre-trained grounding loop recognition model;

[0015] The fifth processing module is used to perform transfer learning processing on the pre-trained grounding loop recognition model to obtain a target grounding loop recognition model.

[0016] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned grounding loop identification method based on deep learning when executing the computer program.

[0017] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned grounding ring identification method based on deep learning are implemented.

[0018] In the scheme implemented by the above-mentioned grounding ring recognition method, device, equipment and medium based on deep learning, based on deep learning, the initial model is lightweighted through the original training data to obtain a lightweight grounding ring recognition model, and the original training data is mixed and enhanced to obtain enhanced training data, and the lightweight grounding ring recognition model is trained through the enhanced training data to obtain a reference grounding ring recognition model, so that small target detection pre-training can be performed based on the reference grounding ring recognition model to obtain a pre-trained grounding ring recognition model, and then the pre-trained grounding ring recognition model can be transferred for learning to obtain a more accurate target grounding ring recognition model, which is conducive to achieving more accurate grounding ring recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0020] Figure 1 It is a schematic diagram of an application environment of a grounding loop identification method based on deep learning in one embodiment of the present invention;

[0021] Figure 2 It is a flowchart of a grounding loop identification method based on deep learning in one embodiment of the present invention;

[0022] Figure 3 is a schematic diagram of a depthwise separable convolution used in a MobileNet network in one embodiment of the present invention;

[0023] Figure 4 is a schematic diagram of the structure of CenterNet in one embodiment of the present invention;

[0024] Figure 5 It is a structural schematic diagram of a grounding loop identification device based on deep learning in one embodiment of the present invention;

[0025] Figure 6 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0026] Figure 7 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] The grounding loop recognition method based on deep learning provided by the embodiment of the present invention can be applied in the following aspects: Figure 1In the application environment, the client communicates with the server through the network. Exemplarily, taking the grounding ring recognition scenario as an example, the target user (R&D personnel or management personnel) can upload the original training data and the initial model related data through the client. Accordingly, the server can obtain the original training data and the initial model related data through the client, and based on deep learning, perform model lightweight processing on the initial model through the original training data to obtain a lightweight grounding ring recognition model, and perform hybrid enhancement processing on the original training data to obtain enhanced training data, and train the lightweight grounding ring recognition model through the enhanced training data to obtain a reference grounding ring recognition model, so that small target detection pre-training can be performed based on the reference grounding ring recognition model to obtain a pre-trained grounding ring recognition model, and then the pre-trained grounding ring recognition model can be transferred to learn and process to obtain a more accurate target grounding ring recognition model, and the recognition result of the target grounding ring recognition model is fed back to the client. Accordingly, the client can receive the recognition result of the target grounding ring recognition model from the server, and can display the recognition result of the target grounding ring recognition model on the client for the target user to query or browse. By adopting the deep learning-based grounding ring identification method provided in the present application, a more accurate target grounding ring identification model can be obtained, and then a more accurate identification result can be identified based on the target grounding ring identification model, which is conducive to achieving more accurate grounding ring identification.

[0029] The client may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server may be implemented by an independent server or a server cluster consisting of multiple servers. The present invention is described in detail below through specific embodiments.

[0030] See also Figure 2 As shown, Figure 2 A flowchart of a grounding loop identification method based on deep learning provided by an embodiment of the present invention includes the following steps:

[0031] S10: Based on deep learning, the initial model is lightweighted to obtain a lightweight grounding loop recognition model.

[0032] Among them, the initial model can be the You Only Look Once-X (YOLO-X) model. Although the YOLO-X model is a relatively lightweight target detection network, its computing requirements are still high for resource-constrained devices such as drones. First, a high-precision teacher model such as YOLO-X can be trained based on deep learning, so that the complete original training data set can be used and the hyperparameters can be adjusted to enable it to accurately identify grounded hanging rings in a variety of environments.

[0033] High temperature parameters are further set to generate soft labels, and the soft labels and original hard labels are used to jointly train a lightweight student model. After the training is completed, the student model is pruned and quantized to optimize, pruning removes redundant neurons or connections, and quantization converts floating-point operations into low-precision integer operations. The optimized student model is then deployed in the embedded hardware of the drone to obtain a lightweight grounding ring recognition model.

[0034] It should be understood that based on deep learning, the initial model is lightweighted to obtain a lightweight grounding ring identification model, which refers to the process of how to obtain a lightweight grounding ring identification model. Among them, step S10, that is, based on deep learning, the initial model is lightweighted to obtain a lightweight grounding ring identification model, including the following steps:

[0035] S11: Based on deep learning, the initial model is trained with the original training data to obtain the grounding loop recognition teacher model;

[0036] S12: Based on the grounding ring recognition teacher model, a soft label is generated by setting a high temperature parameter to obtain soft label data;

[0037] S13: Based on the grounding loop recognition teacher model, a lightweight student model is trained according to the soft label data and the hard label data in the original training data to obtain a grounding loop recognition student model;

[0038] S14: Pruning and quantization optimization processing is performed on the grounding loop recognition student model to obtain a lightweight grounding loop recognition model.

[0039] First, the original training data can be used to train the initial model (such as a relatively large-scale model such as YOLO-X). By continuously adjusting the model hyperparameters on the original training data set, it can be used to accurately identify grounding rings in a variety of environments, thereby obtaining a grounding ring recognition teacher model. This teacher model can achieve good recognition results without resource constraints, thereby providing knowledge guidance for subsequent steps.

[0040] In the process of knowledge distillation, a lightweight student model architecture can be selected, such as a lightweight convolutional neural network architecture, such as MobileNet. MobileNet can be understood as being specially designed for mobile and embedded devices to achieve efficient image classification, target detection and other tasks with limited computing resources and storage. It has fewer parameters and computing requirements and is more suitable for deployment on embedded devices or drones.

[0041] like Figure 3As shown in the figure, the lightweight student model MobileNet has a simple network architecture and can use deep separable convolutions. Compared with traditional convolutional neural networks, it can significantly reduce the number of input channels, thereby effectively reducing the parameters required for the convolution layer. The MobileNet network runs faster than traditional convolutions, has less computational complexity, is easier to implement and deploy on different platforms, and can streamline the computational model, thereby achieving high-precision operations on smaller devices.

[0042] Based on the trained grounding ring recognition teacher model, soft labels can be generated by setting high temperature parameters to obtain soft label data. The soft labels are different from the hard labels in the traditional original training data. The soft labels can contain the confidence distribution information of the teacher model on various targets, that is, not only the most likely category, but also the relationship information between categories, so as to provide richer guidance information for the student model training.

[0043] Among them, the temperature parameter can smooth the probability distribution output by the teacher model to strengthen the relationship information between categories, so that the student model can not only learn the correct classification, but also understand the similarities and differences between different categories, so as to make better classification decisions.

[0044] Furthermore, we can train a lightweight student model architecture (such as MobileNet) based on the soft label data generated by the teacher model and the original hard label data in the original training data based on the ground loop recognition teacher model. During the distillation process, the training goal of the student model is not only to imitate the output of the teacher model, but also to maintain high-precision judgment of the original task.

[0045] Optionally, during the training process, a specific loss function (such as the cross entropy loss between the student model and the true label and the Kullback-Leibler (KL) divergence loss with the soft label of the teacher model) can be used to constrain the learning of the student model so that it can ensure high accuracy for the original task while trying to imitate the output of the teacher model as much as possible, thereby ultimately obtaining a grounded hanging ring recognition student model.

[0046] Optionally, based on the grounding ring recognition teacher model, a lightweight student model training can be performed according to the soft label data and the hard label data in the original training data using the following training loss function to obtain the grounding ring recognition student model:

[0047] L total_x =αL hard +(1-α)L soft

[0048] Among them, L total_xrepresents the training loss function; α represents the hyperparameter; L hard represents the cross entropy loss of the student model for the true label; L soft Represents the Kulbeck-Leibler KL divergence loss between the output of the student model and the teacher model.

[0049] It is understandable that the obtained grounding loop recognition student model can be pruned and quantized for optimization. Pruning can reduce the amount of model calculation by removing redundant neurons or connections in the network and ensure that the model performance is not significantly affected. Quantization can convert the floating point operations of the model into low-precision integer operations (such as 8-bit) to greatly reduce the storage requirements and computational overhead of the model and maintain high accuracy.

[0050] After the student model is trained and optimized through pruning and quantization, it can be deployed in the embedded hardware of the drone. Optionally, further optimizations can be performed based on the performance of the hardware, such as edge computing optimization, that is, specific optimizations of the model for the specific processor architecture of the drone (such as the Advanced RISC Machine (ARM) processor), such as using a lightweight machine learning open source software library (TensorFlow Lite) or an open neural network exchange format (Open Neural Network Exchange, ONNX) and other frameworks to convert the model to a format more suitable for running on mobile devices. Optionally, real-time inference testing can be performed, that is, field testing can be performed on the drone to ensure that the model can run with low latency under actual flight conditions and accurately identify ground loops in complex environments.

[0051] After the above optimization processing, a lightweight grounding ring recognition model can be obtained. The lightweight grounding ring recognition model is more suitable for running on resource-constrained devices (such as embedded devices of drones) to achieve efficient grounding ring recognition tasks.

[0052] S20: Enhanced training data is obtained by performing hybrid enhancement processing on the original training data.

[0053] For target recognition in the UAV ground wire installation and removal mission, this application uses a comprehensive data enhancement technology solution to improve the generalization ability and robustness of the model. This solution can simulate various environmental and condition changes that the UAV may encounter in actual operations by applying multiple random transformations to the original training data.

[0054] Optionally, in response to changes in perspective and posture during the flight of the drone, the present application can perform random geometric transformations on the image, such as rotation, scaling, and translation, so that the model can learn the ability to recognize ground loops at different angles and distances, which is conducive to adapting to changes in perspective and posture during the flight of the drone.

[0055] In order to adapt to different lighting conditions and weather changes, this application can adjust the brightness, contrast and hue of the image so that the model can still accurately identify the target in various lighting environments such as strong light, weak light, cloudy or backlight, so as to cope with different lighting conditions and weather changes.

[0056] This application can add noise and blur effects to images to simulate image quality degradation caused by sensor noise, motion blur, or environmental interference, thereby helping the model maintain stable performance when faced with low-quality or disturbed images.

[0057] The present application can simulate the situation where the target is partially obscured by randomly blocking or erasing parts of the image, thereby enhancing the model's ability to process incomplete information, so that the model can effectively identify the grounding ring in actual scenes even if it is partially obscured by objects such as leaves and wires.

[0058] Optionally, the present application may adopt a hybrid enhancement method to apply different transformation combinations to the same image to further increase the diversity of the data. Through this hybrid enhancement strategy, the model can be exposed to a richer sample distribution during the training process, so that it can have stronger robustness and generalization capabilities in practical applications, thereby ensuring that the drone can accurately complete the grounding ring recognition task in various complex environments.

[0059] S30: Training the lightweight grounding loop recognition model using the enhanced training data to obtain a reference grounding loop recognition model.

[0060] Using the enhanced training data obtained after hybrid enhancement processing to further train the grounding ring recognition model that has completed lightweight processing can improve the performance of the model in various complex environments, enabling it to more accurately and stably identify grounding rings, thereby obtaining a reference grounding ring recognition model.

[0061] Compared with training with only original training data, training based on enhanced training data can significantly improve the generalization ability of the model. When faced with various changes and interferences that may occur in actual application scenarios, the model can more calmly perform accurate recognition, reduce recognition errors caused by changes in environmental factors, and thus improve the success rate and reliability of the entire grounding ring recognition task.

[0062] After the previous model lightweight processing, the model has to some extent the ability to run efficiently on resource-constrained devices. On this basis, by using enhanced training data training, the model's recognition ability is further optimized, so that it can better meet the requirements of accuracy and stability in actual operations while ensuring operating efficiency, thereby obtaining a more practical reference grounding ring recognition model.

[0063] S40: Performing small target detection pre-training based on the reference grounding loop recognition model to obtain a pre-trained grounding loop recognition model.

[0064] The reason for pre-training small target detection based on the reference grounding ring recognition model is that small target detection tasks (such as the detection of grounding rings as small targets) require the model to have richer visual features and stronger feature expression capabilities. By pre-training on a large-scale dataset, the model can learn common small target detection features and patterns, thus laying the foundation for subsequent accurate detection of small targets such as grounding rings.

[0065] In the pre-training stage, the reference grounding ring recognition model is used as the basic model and trained using a large-scale dataset. The training target can be various types of small target detection tasks, such as identifying the location and category of small targets. During the pre-training process, the model will continuously adjust its parameters to adapt to the distribution of small target features in the large-scale dataset. For example, the multi-scale feature extraction module is optimized to better capture the features of small targets at different scales; the prior box (Anchor) design module is improved to adjust the size, shape and distribution strategy of the preset box according to the size and shape distribution of small targets in the dataset.

[0066] It should be understood that pre-training small target detection based on the reference grounding loop recognition model to obtain a pre-trained grounding loop recognition model refers to the process of pre-training on a large-scale data set to obtain a pre-trained grounding loop recognition model. Among them, step S40, that is, pre-training small target detection based on the reference grounding loop recognition model to obtain a pre-trained grounding loop recognition model, includes the following steps:

[0067] S41: Based on a multi-scale feature extraction network, the reference grounding loop recognition model is improved to obtain a multi-scale grounding loop recognition model;

[0068] S42: Based on the improved prior frame Anchor design, the multi-scale grounding ring recognition model is adjusted for small target detection to obtain an improved grounding ring recognition model;

[0069] S43: Based on the super-resolution network, the detection performance of the improved grounding ring recognition model is enhanced to obtain an enhanced grounding ring recognition model;

[0070] S44: Using the improved loss function, the enhanced grounding loop recognition model is trained to obtain a pre-trained grounding loop recognition model.

[0071] Due to the small size of small targets, their features are usually lost in deeper network layers due to over-downsampling. Therefore, in order to ensure that small targets can still be clearly represented on higher-resolution feature maps, multi-scale feature extraction is crucial. This application adopts a feature pyramid network (FPN) structure to fuse feature maps of different resolutions through a top-down feature pyramid structure. Among them, FPN can utilize shallow features (higher resolution, containing more spatial information) and deep features (lower resolution, but with stronger semantic information) to achieve the purpose of small target detection at multiple scales.

[0072] On the basis of the original FPN, the resolution of the feature map can be further increased to adopt more low-level feature maps to capture small target information, so that the model can focus on the details of small targets on the high-resolution feature map and combine the deep semantic information for more accurate classification, thereby obtaining a multi-scale grounded loop recognition model.

[0073] Traditional target detection methods usually use preset anchors to locate targets. Therefore, the size design of anchors is crucial for the detection of small targets. If the size of the anchor is too large, the small target may not be accurately captured; if the size of the anchor is too small, it is easily affected by background noise. During training, the appropriate anchor size can be set according to the actual size of the small target. For example, by analyzing the average size of small targets in the data set, the width-to-height ratio and scale of the anchor can be adjusted to make it more suitable for small target detection.

[0074] This application adopts the Anchor-free strategy, which can be understood as no longer relying on preset anchors to locate the target, but achieving target positioning by predicting the key points or center points of the target object. For small targets, the Anchor-free strategy can avoid the problem of mismatched anchor settings, and the Anchor-free strategy is more flexible and can better adapt to the various forms and distributions of small targets. For example, Figure 4As shown in the figure, in actual application, the effect of small target detection using the Anchor-free detection framework of the Center-point based Object Detection Network (CenterNet) is better than the traditional Anchor-based small target detection. The Anchor-free detection framework of CenterNet can more accurately locate and identify small targets.

[0075] Furthermore, since small targets (such as grounding rings) are small in size, the details presented in the original image may not be clear enough, which makes it difficult for the model to accurately recognize them. By processing the improved grounding ring recognition model through a super-resolution network, the image resolution on which the model is based can be improved, so that small targets can show their detailed features more clearly in the image, thereby enhancing the model's detection performance for small targets.

[0076] Super-resolution networks can use a series of advanced algorithms and specific network structures to improve image resolution. By training a super-resolution network, small targets in the input image can be enlarged and their pixel information can be increased, making the small targets easier to be captured by the detection network. The super-resolution network can analyze and process the input image, and by learning the features and patterns of the image, it can intelligently increase the pixel information of the image, thereby increasing the resolution of the image from a lower level to a higher level.

[0077] When the improved grounding ring recognition model is connected to the super-resolution network, the image input to the model will be processed by the super-resolution network first during the model's grounding ring small target detection process, so that the edge, shape, texture and other details of the small target can be obtained more clearly. For example, the originally blurred outline of the grounding ring and the details of the hook can be more accurately captured by the model after the resolution is improved, so that the model can more accurately locate and identify the small grounding ring target, and finally obtain an enhanced grounding ring recognition model.

[0078] In the object detection task, the loss function plays a key role and can be used to measure the difference between the model's prediction results and the actual situation. For small object detection, conventional loss functions (such as standard cross entropy or mean square error loss functions) may not be well adapted to its characteristics, which may result in the model being unable to effectively optimize for small objects during training.

[0079] The present application adopts an improved loss function to train the enhanced grounding loop recognition model, so that the model can more accurately adjust its own parameters in the direction of reducing this difference during the training process, so as to better adapt to small target detection tasks, improve the model's prediction accuracy for small targets, and finally obtain a pre-trained grounding loop recognition model.

[0080] Optionally, the present application may implement the step of using the improved loss function to train the enhanced grounding loop recognition model to obtain the pre-trained grounding loop recognition model by using the improved loss function shown in the following formula:

[0081]

[0082] FL(p ti )=-β ti (1-p ti ) γ log(p ti )

[0083]

[0084] Among them, L total_ad is the improved loss function; N is the number of targets; i is the target index; FL(p ti ) is the classification loss function; p ti is the predicted probability of the category of the i-th target; λ is the weight parameter that balances the importance of classification loss and positioning loss; L GIoU (A i ,B i ) is the positioning loss function; A i is the predicted bounding box of the i-th target; B i is the true bounding box of the i-th target; β ti is the weight parameter for balancing positive and negative samples; γ is the adjustment factor; log is the logarithmic operation; IoU i C is the intersection and union ratio; i To include A i and B i The minimum enclosing rectangle of .

[0085] It should be noted that the classification loss function of the present application is illustrated by using an improved loss function (Focal Loss) as an example to deal with the category imbalance problem in small target detection. The Focal Loss can pay more attention to small targets that are difficult to detect and reduce the negative impact of simple backgrounds, thereby improving the detection accuracy of small targets. For the bounding box regression task, an improved intersection over union loss function (Generalized Intersection over Union Loss, GIoU Loss) is used, which is the positioning loss function of the present application. The GIoU Loss can be used to measure the difference between the predicted bounding box and the true bounding box in target detection. It pays more attention to the degree of overlap of the bounding box than the traditional mean square error loss function, which helps to improve the positioning accuracy of small targets. It can be understood that accurate bounding box positioning is crucial, especially when the target is very small.

[0086] Optionally, data augmentation can help the model learn how to better recognize small objects by generating more diverse scenes of small objects. For example, randomly cropping the area where the small object is located in the image and appropriately enlarging it can help enhance the model's attention to small objects. When scaling the image, avoid shrinking the small object to the point where it cannot be recognized.

[0087] Among them, "mixup" is a commonly used data enhancement technology in the field of data enhancement for deep learning, especially in classification and regression tasks, which can improve the generalization ability of deep learning models. By linearly combining different samples to enhance the training diversity of the model, it can effectively reduce overfitting and improve the robustness of the model in complex tasks. The basic idea of ​​mixup is to mix two images and their corresponding labels in a certain proportion to generate a new training sample, that is, not only mix in the input space, but also linearly mix in the label space, so that the model can learn a wider data distribution.

[0088] S50: performing transfer learning processing on the pre-trained grounding loop recognition model to obtain a target grounding loop recognition model.

[0089] Further transfer learning of the pre-trained grounding ring recognition model can make the model accurately adapt to the specific scenarios and task requirements of grounding ring small target detection, further improve the detection accuracy, and thus obtain a more accurate target grounding ring recognition model, so as to achieve good results in practical applications (such as grounding ring recognition operations carried out by drones). Using the transfer learning method, the pre-trained model is fine-tuned to the specific scenarios and data sets of small target detection tasks, especially for scenes shot by drones, which can reduce the dependence on training data and improve the generalization ability of small target detection.

[0090] When migrating the pre-trained model to the grounded loop small target detection task, some of the model's feature extraction layers can be frozen first. This is because these layers have learned relatively common small target features during the pre-training process, and freezing them can prevent these useful features from being destroyed during the fine-tuning process. Then, the latter layers of the model (such as the classification layer, regression layer, etc.) are fine-tuned to adapt them to the specific categories and task requirements of grounded loop small target detection. For example, a special classification layer parameter adjustment is performed for the grounded loop category so that the model can accurately distinguish between grounded loops and other possible targets.

[0091] Adjust the loss function according to the specific requirements of small target detection of ground loops. For example, in the small target position regression, the weight of the small target position accuracy can be increased, because the accuracy of the position detection of ground loops as small targets is crucial for practical applications (such as drone operations). At the same time, for the loss function of the category prediction part, targeted adjustments can be made according to the possible misclassification of ground loops in actual scenarios (such as confusion with other similar small targets), guiding the model to more accurately identify ground loops.

[0092] Fine-tune using datasets specially collected for ground loop small target detection (can be datasets containing ground loops taken by actual drones). These datasets contain various postures, lighting, occlusion, etc. of ground loops in actual scenes. By fine-tuning on these data, the model can better adapt to the special scenes and requirements of ground loop small target detection tasks, and further improve detection accuracy.

[0093] This application obtains a model that can accurately identify grounding rings by lightweighting the model, enhancing robustness, and detecting small targets with high precision, thereby meeting the necessary conditions for the task of installing and removing ground wires on drones. This application can reduce the computational complexity and power consumption of the model by performing network pruning and distillation optimization on the YOLO-X model, thereby achieving real-time deployment on the drone side. This application can improve the recognition accuracy and adaptability of the model in complex environments through data enhancement technology (such as geometric transformation, lighting adjustment, random occlusion, etc.). This application uses a multi-scale feature extraction network, an improved Anchor design, and an improved loss function to solve the problem that the ring is difficult to detect as a small target at a long distance, thereby ensuring that drones can efficiently and accurately identify grounding rings in a variety of complex scenarios.

[0094] It can be seen that in the above scheme, based on deep learning, the initial model is lightweighted through the original training data to obtain a lightweight grounding ring recognition model, and the original training data is mixed and enhanced to obtain enhanced training data, so that the lightweight grounding ring recognition model is trained through the enhanced training data to obtain a reference grounding ring recognition model, so that small target detection pre-training can be performed based on the reference grounding ring recognition model to obtain a pre-trained grounding ring recognition model, and then the pre-trained grounding ring recognition model can be transferred for learning to obtain a more accurate target grounding ring recognition model, which is conducive to achieving more accurate grounding ring recognition.

[0095] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0096] In one embodiment, a grounding ring identification device based on deep learning is provided, and the grounding ring identification device based on deep learning corresponds to the grounding ring identification method based on deep learning in the above embodiment. Figure 5 As shown, the grounding loop identification device based on deep learning includes a first processing module 101, a second processing module 102, a third processing module 103, a fourth processing module 104 and a fifth processing module 105. The functional modules are described in detail as follows:

[0097] The first processing module is used to perform model lightweight processing on the initial model through original training data based on deep learning to obtain a lightweight grounding loop recognition model;

[0098] The second processing module is used to obtain enhanced training data by performing mixed enhancement processing on the original training data;

[0099] A third processing module is used to train the lightweight grounding loop recognition model through the enhanced training data to obtain a reference grounding loop recognition model;

[0100] A fourth processing module is used to perform small target detection pre-training based on the reference grounding loop recognition model to obtain a pre-trained grounding loop recognition model;

[0101] The fifth processing module is used to perform transfer learning processing on the pre-trained grounding loop recognition model to obtain a target grounding loop recognition model.

[0102] In one embodiment, the first processing module 101 is used to perform model lightweight processing on the initial model through original training data based on deep learning to obtain a lightweight grounding loop recognition model, specifically for:

[0103] Based on deep learning, the initial model is trained with the original training data to obtain the grounding loop recognition teacher model;

[0104] Based on the grounding ring recognition teacher model, a soft label is generated by setting a high temperature parameter to obtain soft label data;

[0105] Based on the grounding loop recognition teacher model, a lightweight student model is trained according to the soft label data and the hard label data in the original training data to obtain a grounding loop recognition student model;

[0106] The grounding loop recognition student model is pruned and quantized to optimize and obtain a lightweight grounding loop recognition model.

[0107] In one embodiment, the first processing module 101 is used to perform lightweight student model training based on the grounding loop recognition teacher model and the hard label data in the soft label data and the original training data to obtain the grounding loop recognition student model, specifically for:

[0108] Based on the grounding ring recognition teacher model, according to the soft label data and the hard label data in the original training data, the following training loss function is used to perform lightweight student model training to obtain the grounding ring recognition student model:

[0109] L total_x =αL hard +(1-α)L soft

[0110] Among them, L total_x represents the training loss function; α represents the hyperparameter; L hard represents the cross entropy loss of the student model for the true label; L soft Represents the Kulbeck-Leibler KL divergence loss between the output of the student model and the teacher model.

[0111] In one embodiment, the fourth processing module 104 is used to perform small target detection pre-training based on the reference grounding loop recognition model to obtain a pre-trained grounding loop recognition model, which is specifically used to:

[0112] Based on a multi-scale feature extraction network, the reference grounding loop recognition model is improved to obtain a multi-scale grounding loop recognition model;

[0113] Based on the improved prior frame Anchor design, the multi-scale grounding ring recognition model is adjusted for small target detection to obtain an improved grounding ring recognition model;

[0114] Based on a super-resolution network, the detection performance of the improved grounding ring recognition model is enhanced to obtain an enhanced grounding ring recognition model;

[0115] The enhanced grounding loop recognition model is trained using the improved loss function to obtain the pre-trained grounding loop recognition model.

[0116] In one embodiment, the fourth processing module 104 is used to train the enhanced grounding loop recognition model using an improved loss function to obtain a pre-trained grounding loop recognition model, which is specifically used to:

[0117] The improved loss function shown in the following formula is used to implement the step of using the improved loss function to train the enhanced grounding loop recognition model to obtain the pre-trained grounding loop recognition model:

[0118]

[0119] FL(p ti )=-β ti (1-p ti ) γ log(p ti )

[0120]

[0121] Among them, L total_ad is the improved loss function; N is the number of targets; i is the target index; FL(p ti ) is the classification loss function; p ti is the predicted probability of the category of the i-th target; λ is the weight parameter that balances the importance of classification loss and positioning loss; L GIoU (A i ,B i ) is the positioning loss function; A i is the predicted bounding box of the i-th target; B i is the true bounding box of the i-th target; β ti is the weight parameter for balancing positive and negative samples; γ is the adjustment factor; IoU i C is the intersection and union ratio; i To include A i and B i The minimum enclosing rectangle of .

[0122] The present invention provides a grounding ring recognition device based on deep learning. Based on deep learning, the initial model is lightweighted through original training data to obtain a lightweight grounding ring recognition model, and the original training data is mixed and enhanced to obtain enhanced training data. The lightweight grounding ring recognition model is trained through the enhanced training data to obtain a reference grounding ring recognition model, so that small target detection pre-training can be performed based on the reference grounding ring recognition model to obtain a pre-trained grounding ring recognition model, and then the pre-trained grounding ring recognition model can be transferred and learned to obtain a more accurate target grounding ring recognition model, which is conducive to achieving more accurate grounding ring recognition.

[0123] For the specific definition of the grounding ring identification device based on deep learning, please refer to the definition of the grounding ring identification method based on deep learning above, which will not be repeated here. Each module in the above-mentioned grounding ring identification device based on deep learning can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0124] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, the functions or steps on the service side of a grounding hanging ring identification method based on deep learning are implemented.

[0125] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, the functions or steps on the client side of a grounding hanging ring identification method based on deep learning are implemented.

[0126] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0127] Based on deep learning, the initial model is lightweighted through the original training data to obtain a lightweight grounding loop recognition model;

[0128] Enhanced training data is obtained by performing mixed enhancement processing on the original training data;

[0129] The lightweight grounding loop recognition model is trained by using the enhanced training data to obtain a reference grounding loop recognition model;

[0130] Performing small target detection pre-training based on the reference grounding loop recognition model to obtain a pre-trained grounding loop recognition model;

[0131] The pre-trained grounding loop recognition model is subjected to transfer learning processing to obtain a target grounding loop recognition model.

[0132] The present invention provides a computer device, which is based on deep learning. The initial model is lightweighted through original training data to obtain a lightweight grounding ring recognition model, and the original training data is mixed and enhanced to obtain enhanced training data. The lightweight grounding ring recognition model is trained through the enhanced training data to obtain a reference grounding ring recognition model, so that small target detection pre-training can be performed based on the reference grounding ring recognition model to obtain a pre-trained grounding ring recognition model, and then the pre-trained grounding ring recognition model can be transferred and learned to obtain a more accurate target grounding ring recognition model, which is conducive to achieving more accurate grounding ring recognition.

[0133] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0134] Based on deep learning, the initial model is lightweighted through the original training data to obtain a lightweight grounding loop recognition model;

[0135] Enhanced training data is obtained by performing mixed enhancement processing on the original training data;

[0136] The lightweight grounding loop recognition model is trained by using the enhanced training data to obtain a reference grounding loop recognition model;

[0137] Performing small target detection pre-training based on the reference grounding loop recognition model to obtain a pre-trained grounding loop recognition model;

[0138] The pre-trained grounding loop recognition model is subjected to transfer learning processing to obtain a target grounding loop recognition model.

[0139] The present invention provides a computer-readable storage medium, which is based on deep learning and performs model lightweight processing on an initial model through original training data to obtain a lightweight grounding ring recognition model, and performs mixed enhancement processing on the original training data to obtain enhanced training data, so as to train the lightweight grounding ring recognition model through the enhanced training data to obtain a reference grounding ring recognition model, so that small target detection pre-training can be performed based on the reference grounding ring recognition model to obtain a pre-trained grounding ring recognition model, and then the pre-trained grounding ring recognition model can be subjected to transfer learning processing to obtain a more accurate target grounding ring recognition model, which is conducive to achieving more accurate grounding ring recognition.

[0140] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0141] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0142] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0143] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A grounding loop recognition method based on deep learning, characterized in that: The grounding loop recognition method based on deep learning includes: Based on deep learning, the initial model is lightweighted through the original training data to obtain a lightweight grounding loop recognition model; Enhanced training data is obtained by performing mixed enhancement processing on the original training data; The lightweight grounding loop recognition model is trained by using the enhanced training data to obtain a reference grounding loop recognition model; Performing small target detection pre-training based on the reference grounding loop recognition model to obtain a pre-trained grounding loop recognition model; The pre-trained grounding loop recognition model is subjected to transfer learning processing to obtain a target grounding loop recognition model.

2. The grounding loop identification method based on deep learning according to claim 1 is characterized in that: Based on deep learning, the initial model is lightweighted by original training data to obtain a lightweight grounding ring recognition model, including: Based on deep learning, the initial model is trained with the original training data to obtain the grounding loop recognition teacher model; Based on the grounding ring recognition teacher model, a soft label is generated by setting a high temperature parameter to obtain soft label data; Based on the grounding loop recognition teacher model, a lightweight student model is trained according to the soft label data and the hard label data in the original training data to obtain a grounding loop recognition student model; The grounding loop recognition student model is pruned and quantized to optimize and obtain a lightweight grounding loop recognition model.

3. The grounding loop identification method based on deep learning according to claim 2 is characterized in that: The method of performing lightweight student model training based on the grounding loop recognition teacher model according to the soft label data and the hard label data in the original training data to obtain the grounding loop recognition student model includes: Based on the grounding ring recognition teacher model, according to the soft label data and the hard label data in the original training data, the following training loss function is used to perform lightweight student model training to obtain the grounding ring recognition student model: L total_x =αL hard +(1-α)L soft Among them, L total_x represents the training loss function; α represents the hyperparameter; L hard represents the cross entropy loss of the student model for the true label; L soft Represents the Kulbeck-Leibler KL divergence loss between the output of the student model and the teacher model.

4. The grounding loop identification method based on deep learning according to claim 3 is characterized in that: The pre-training of small target detection based on the reference grounding loop recognition model to obtain a pre-trained grounding loop recognition model includes: Based on a multi-scale feature extraction network, the reference grounding loop recognition model is improved to obtain a multi-scale grounding loop recognition model; Based on the improved prior frame Anchor design, the multi-scale grounding ring recognition model is adjusted for small target detection to obtain an improved grounding ring recognition model; Based on a super-resolution network, the detection performance of the improved grounding ring recognition model is enhanced to obtain an enhanced grounding ring recognition model; The enhanced grounding loop recognition model is trained using the improved loss function to obtain the pre-trained grounding loop recognition model.

5. The grounding loop identification method based on deep learning according to claim 4 is characterized in that: The improved loss function is used to train the enhanced grounding loop recognition model to obtain a pre-trained grounding loop recognition model, including: The improved loss function shown in the following formula is used to implement the step of using the improved loss function to train the enhanced grounding loop recognition model to obtain the pre-trained grounding loop recognition model: FL(p ti )=-β ti (1-p ti ) γ log(p ti ) Among them, L total_ad is the improved loss function; N is the number of targets; i is the target index; FL(p ti ) is the classification loss function; p ti is the predicted probability of the category of the i-th target; λ is the weight parameter that balances the importance of classification loss and positioning loss; L GIoU (A i , B i ) is the positioning loss function; A i is the predicted bounding box of the i-th target; B i is the true bounding box of the i-th target; β ti is the weight parameter for balancing positive and negative samples; γ is the adjustment factor; log is the logarithmic operation; IoU i C is the intersection ratio; i To include A i and B i The minimum enclosing rectangle of .

6. A grounding ring identification device based on deep learning, characterized in that: The grounding loop identification device based on deep learning includes: The first processing module is used to perform model lightweight processing on the initial model through original training data based on deep learning to obtain a lightweight grounding loop recognition model; The second processing module is used to obtain enhanced training data by performing mixed enhancement processing on the original training data; A third processing module is used to train the lightweight grounding loop recognition model through the enhanced training data to obtain a reference grounding loop recognition model; A fourth processing module is used to perform small target detection pre-training based on the reference grounding loop recognition model to obtain a pre-trained grounding loop recognition model; The fifth processing module is used to perform transfer learning processing on the pre-trained grounding loop recognition model to obtain a target grounding loop recognition model.

7. The grounding loop identification device based on deep learning according to claim 6 is characterized in that: The first processing module is used to perform model lightweight processing on the initial model through original training data based on deep learning to obtain a lightweight grounding loop recognition model, specifically for: Based on deep learning, the initial model is trained with the original training data to obtain the grounding loop recognition teacher model; Based on the grounding ring recognition teacher model, a soft label is generated by setting a high temperature parameter to obtain soft label data; Based on the grounding loop recognition teacher model, a lightweight student model is trained according to the soft label data and the hard label data in the original training data to obtain a grounding loop recognition student model; The grounding loop recognition student model is pruned and quantized to optimize and obtain a lightweight grounding loop recognition model.

8. The grounding loop identification device based on deep learning according to claim 7 is characterized in that: The first processing module is used to perform lightweight student model training based on the grounding loop recognition teacher model and the hard label data in the soft label data and the original training data to obtain the grounding loop recognition student model, and is specifically used to: Based on the grounding ring recognition teacher model, according to the soft label data and the hard label data in the original training data, the following training loss function is used to perform lightweight student model training to obtain the grounding ring recognition student model: L total_x =αL hard +(1-α)L soft Among them, L total_x represents the training loss function; α represents the hyperparameter; L hard represents the cross entropy loss of the student model for the true label; L soft Represents the Kulbeck-Leibler KL divergence loss between the output of the student model and the teacher model.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the grounding loop identification method based on deep learning as described in any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the grounding loop identification method based on deep learning as described in any one of claims 1 to 5 is implemented.

Images