Target detection method, system and equipment based on improved twin network, and medium
Through an improved twin network, combined with EfficientNet and YOLOv8 backbone, and introducing EMA attention mechanism and Ghost module, the problem of insulator target detection in complex power line environments is solved, and efficient and accurate target detection effect is achieved.
Patent Information
- Application Number
- CN202411811480.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-10
- Publication Date
- 2025-03-28
AI Technical Summary
In complex power line environments, it is difficult for the drone vision system to accurately detect the target of insulators, due to dense vegetation, building shading, changing weather conditions and uncertainty in the light angle. Insulators have a variety of shapes, reflective materials, adhered foreign objects and potential damage patterns, which increase the detection difficulty.
Using an improved twin network, combined with EfficientNet and YOLOv8 backbone, feature images of different scales are extracted by introducing EMA attention mechanism and Ghost module, and fusion and adjustment are carried out through feature pyramid network to generate richer feature representations to improve the accuracy of target detection.
It improves the feature expression ability of the model in complex environments, enhances the detection accuracy and efficiency of insulator targets, and reduces the consumption of computing resources.
Smart Images
Figure CN119723388A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of target detection technology, and more specifically, to a target detection method, system, device and medium based on an improved twin network. Background Art
[0002] In modern society, the stable operation of the power system is crucial to economic development and social life, and the safe maintenance of power lines has become one of the key links to ensure the stability of the power system. As an important component of power transmission lines, insulators not only need to withstand the test of high voltage currents, but also withstand the erosion of the natural environment (such as bird droppings of raptors in some areas of the plateau attached to insulators, causing bird droppings-related bird-related failures). Their status is directly related to the normal operation of transmission lines and the safe operation of power grids. Traditional insulator inspections usually rely on manual visual inspections on towers, which is not only inefficient, but also poses a great safety hazard in severe weather or complex terrain conditions.
[0003] The rise of drone technology has brought revolutionary changes to power inspection. Drones can not only easily reach areas that are difficult to reach by human labor, but also continue to operate in complex environments. The high-definition cameras and advanced image processing technologies they carry provide unprecedented possibilities for remote detection of insulators. However, technical challenges also come with it. The complexity of the environment around power lines, such as dense vegetation, building obstructions, unpredictable weather conditions, and uncertainty in lighting angles, all pose huge challenges to the detection of insulator targets by drone vision systems; in addition, the diversity of insulator shapes, the reflective properties of surface materials, foreign objects such as bird droppings attached to the surface, and potential damage forms also increase the recognition difficulty of the detection algorithm. Summary of the invention
[0004] The purpose of the present invention is to provide a target detection method, system, device and medium based on an improved twin network, taking into account the multiple challenges in the complex environment of power lines, aiming to achieve efficient and accurate target detection.
[0005] The above technical objectives of the present invention are achieved through the following technical solutions:
[0006] In a first aspect, the present application provides a target detection method based on an improved twin network, comprising the following specific steps:
[0007] Preprocess several sample images taken by the drone, and construct a data set for model training through the preprocessed sample images, the data set includes a training set, a validation set, and a test set;
[0008] The training set is input into the backbone network of the detection model of the improved twin EfficientNet-YOLOv8 network, and feature images of different scales are extracted from several sample images of the training set through the introduced EMA mechanism and Ghost module;
[0009] Each feature image of different scales is fused and adjusted to generate feature representation, and the target prediction result in each sample image is obtained through the output layer prediction of the detection model. The target prediction result includes the target position, target size and confidence score;
[0010] Based on the improved loss function, the prediction performance of the detection model is evaluated using the validation set, and the detection model is adjusted using each evaluation result and the loss function value until the detection model reaches the end of training condition;
[0011] The images to be tested taken by the drone are preprocessed, and the preprocessed images to be tested are input into the detection model that meets the training end conditions for processing to obtain the target detection results of the images to be tested.
[0012] Based on the above technical solution, the present invention can also be improved as follows.
[0013] Furthermore, the above-mentioned preprocessing of several sample images taken by the drone is specifically as follows:
[0014] Plan the flight path and shooting strategy of the drone according to the distribution of power lines and inspection requirements, take target images of insulators in different time periods and weather conditions, and preliminarily screen target images that meet the preset conditions;
[0015] Adjusting image parameters of a target image that meets preset conditions, transforming the target image after the image parameters are adjusted, and determining a sample image from the transformed target image and the target image after the image parameters are adjusted;
[0016] The detection targets in the sample images of uniform size are annotated to obtain a data set, and the data set is divided according to a preset ratio to obtain a training set, a validation set, and a test set included in the data set.
[0017] Furthermore, in the improved twin EfficientNet-YOLOv8 network, the standard convolution of the convolution layer in the YOLOv8 network is replaced with a depthwise separable convolution, where: the depthwise separable convolution is applied to each input channel, and the pointwise convolution in the YOLOv8 network integrates information across channels; the fully connected layer of the SE module in the EfficientNet network is replaced with a 1×1 convolution layer that realizes cross-channel information aggregation; an EMA attention mechanism and a Ghost module are added to the connection module SPPF between the end of the YOLOv8 network and the neck network, where:
[0018] The training set is input into the backbone network of the improved twin EfficientNet-YOLOv8, where the same-size feature maps generated by the EfficientNet network and the YOLOv8 network in each layer module are represented as follows:
[0019] (p1 EfficientNet 、p2 EfficientNet 、p3 EfficientNetf )、(p1 yolov8 、p2 yolov8 、p3 yolov8 );
[0020] The fusion features of the same layer are represented as (p1, p2, p3), and the feature processing method is expressed as:
[0021] p1=conv(1×1)(concatenate(p1 EfficientNet ,p1 yolov8 )); where concatenate() is the channel concatenation function;
[0022] The processing of fusion features is as follows:
[0023] Each fusion feature is expressed as: X∈R H×W×C , and divided into T cross-channel sub-features, the input features of each fusion feature are: X = [X0, X1, ... X t ,...,X T-1 ], the result of feature pooling is:
[0024] Where: H, W, C represent the length, width, and channel dimensions of each feature map, respectively, P C H (H), P C W (W) represents the feature pooling processing results in horizontal width and vertical height dimensions respectively.
[0025] Furthermore, the above method also includes:
[0026] Based on the result of feature pooling, the basic feature map is generated by conventional convolution in the Ghost module, and is expanded into a Ghost feature map based on a linear transformation with low computational complexity, and the Ghost feature map is determined as a feature image. The specific calculation method is as follows:
[0027] Y = X × f + b, where Y is the basic feature map obtained by conventional convolution operation; f is the convolution filter; b is the bias term, and X represents the fused feature after feature pooling processing;
[0028] Y ij =ψ i,j (Y i ), where Y ij is the Ghost feature map after linear transformation expansion, ψ i,j is a linear processing function, Y i represents the i-th feature map in the basic feature map, and j is the j-th linear operation performed on the basic feature map.
[0029] Furthermore, the above-mentioned feature images of different scales are fused and adjusted to generate feature representations, specifically:
[0030] Utilizing the bottom-up and top-down structures, feature images of different scales are fused and adjusted based on the feature pyramid network, and the corresponding feature representations are obtained.
[0031] Furthermore, the target prediction results in each sample image are obtained by predicting the output layer of the detection model, specifically:
[0032] Information is extracted from each feature representation through two 3×3 convolutions and one 1×1 convolution, and the original prediction value is output as the center coordinates, width and height of the bounding box, as well as the category confidence score to obtain the target prediction result.
[0033] Furthermore, the above loss function is specifically:
[0034] L total =αL cls +βL DFL +δL CIOU +(1-α-β-δ)L sia ,in:
[0035]
[0036] Where, L total Represents the loss function value, L cls is the classification loss, L DFL is the classification focal loss, L CIOU is the complete border loss, α, β, δ are weight coefficients, and α, β, δ∈[0,1], L siais the matching loss of the detection model, N represents the number of training samples in the training set, m is the distance threshold of dissimilar samples, Y is the label of whether two samples match, Y = 1 means the two samples are similar or matched, Y = 0 means the two samples do not match, E w Represents the similarity value between two training samples.
[0037] In a second aspect, the present application provides a target detection system based on an improved twin network, which is applied to any target detection method based on an improved twin network in the first aspect, including:
[0038] The first module is used to preprocess a number of sample images taken by the drone, and construct a data set for model training through the preprocessed sample images, the data set includes a training set, a validation set and a test set;
[0039] The second module is used to input the training set into the backbone network of the detection model of the improved twin EfficientNet-YOLOv8 network, and extract feature images of different scales from several sample images of the training set through the introduced EMA mechanism and Ghost module;
[0040] The third module is used to fuse and adjust the feature images of different scales and generate feature representations. The target prediction results in each sample image are obtained through the output layer prediction of the detection model. The target prediction results include target position, target size and confidence score.
[0041] The fourth module is used to evaluate the prediction performance of the detection model using the validation set based on the improved loss function, and adjust the detection model using each evaluation result and the loss function value until the detection model reaches the end of training condition;
[0042] The fifth module is used to preprocess the test images taken by the drone, and input the preprocessed test images into the detection model that meets the training end conditions for processing to obtain the target detection results of the test images.
[0043] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any one of the methods in the first aspect when executing the computer program.
[0044] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions enable a computer to execute any one of the methods in the first aspect.
[0045] Compared with the prior art, the present invention has at least the following beneficial effects:
[0046] 1. Adopt a twin network that combines EfficientNet and YOLOv8 backbones to enhance the feature extraction capability of the backbone network, generate richer feature representations with multi-level spatial structures, and improve the feature expression capability of the model in complex networks;
[0047] 2. By using depth-separable convolution instead of standard convolution, the model's parameters and computational requirements are reduced, making the model more lightweight. Depth-wise convolution focuses on information processing for each input channel, while point-by-point convolution integrates information across channels, which helps maintain the depth and width of the model while reducing resource consumption;
[0048] 3. Add the EMA attention mechanism at the end of the YOLOv8 backbone network to dynamically adjust the channel weights, promote information interaction between the two sub-networks, effectively integrate information, avoid information loss caused by dimensionality reduction, and control the growth of computational complexity, thereby improving the overall performance of the model;
[0049] 4. The Ghost module expands the feature map through low-cost linear transformation, which can reproduce and enrich the features extracted by the initial convolutional layer at a lower computational complexity, effectively reducing the computational burden brought by a large number of convolution kernels and improving the performance of the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:
[0051] Figure 1 A method flow chart of the detection method in an embodiment of the present invention;
[0052] Figure 2 Schematic diagram of a twin network of EfficientNet and YOLOv8 backbone in an embodiment of the present invention;
[0053] Figure 3 It is a schematic diagram of part of the twin network in an embodiment of the present invention;
[0054] Figure 4 A connection diagram of a detection system in an embodiment of the present invention;
[0055] Figure 5 Schematic diagram of the connection of electronic equipment in an embodiment of the present invention. DETAILED DESCRIPTION
[0056] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.
[0057] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0058] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0059] In the description of the embodiments of the present invention, "plurality" means at least 2.
[0060] Embodiment 1: Considering the multiple challenges in the complex environment of power lines, the purpose is to achieve efficient and accurate target detection. To achieve the above purpose, the embodiment of the present application provides a target detection method based on an improved twin network, such as Figure 1 As shown, the following specific steps are included:
[0061] S1, preprocessing a number of sample images taken by the drone, and constructing a data set for model training through the preprocessed sample images, the data set includes a training set, a validation set and a test set.
[0062] Optionally, the preprocessing of the sample images taken by the drone is specifically as follows:
[0063] S11, plans the flight path and shooting strategy of the drone according to the distribution of power lines and inspection requirements, shoots target images of insulators under different time periods and weather conditions, and preliminarily screens target images that meet preset conditions.
[0064] Among them, according to the distribution of power lines and inspection needs, the drone flight path and shooting strategy are planned to ensure that the area required for inspection is covered; at the same time, target images such as insulators are taken at different time periods and weather conditions, and clear, unobstructed, and appropriately angled images are preliminarily screened out, and images that are too dark, too bright, or blurred are removed to ensure the quality of the data set.
[0065] S12, adjusting image parameters of a target image that meets preset conditions, transforming the target image after the image parameters are adjusted, and determining a sample image from the transformed target image and the target image after the image parameters are adjusted.
[0066] Among them, image details can be enhanced by adjusting parameters such as brightness, contrast, and sharpness, and sample images can be transformed by flipping, blocking, scaling, and other transformations to enrich sample data.
[0067] S13, annotating the detection targets in the sample images of uniform size to obtain a data set, and dividing the data set according to a preset ratio to obtain a training set, a validation set and a test set included in the data set.
[0068] Specifically, the sample size needs to be unified. Semi-automatic tools can be used to annotate the detection targets in the samples, including the target location and category. The preprocessed sample data is divided into training set, validation set and test set at a ratio of 7:2:1. The training set, validation set and test set are used for model training, model parameter adjustment and model performance evaluation, respectively.
[0069] S2, the training set is input into the backbone network of the detection model of the improved twin EfficientNet-YOLOv8 network, and feature images of different scales are extracted from several sample images of the training set through the introduced EMA mechanism and Ghost module.
[0070] Optionally, the above is an improved twin EfficientNet-YOLOv8 network, wherein the structure of the improved twin EfficientNet-YOLOv8 network is as follows Figure 2 and Figure 3 As shown in the figure: the standard convolution of the convolutional layer in the YOLOv8 network is replaced with a depthwise separable convolution, where: the depthwise separable convolution is applied to each input channel, and the point-by-point convolution in the YOLOv8 network integrates information across channels; the fully connected layer of the SE module in the EfficientNet network is replaced with a 1×1 convolution layer that realizes cross-channel information aggregation; a layer of EMA attention mechanism and Ghost module are added to the connection module SPPF between the end of the YOLOv8 network and the neck network.
[0071] Among them, Figure 2 and Figure 3As shown in the figure, the training set obtained in step S1 is input into the backbone network of the improved twin EfficientNet-YOLOv8, and features of different scales are extracted through multi-layer convolution modules. Continuous upsampling operations capture high-level features to obtain spatial structure information at different levels of the feature map. In the twin network structure, the original sample data is input into the backbone networks of EfficientNet and YOLOv8 respectively. In the process of feature extraction of these two networks, the forward processing methods of the data are independent of each other. The feature maps output by each layer are spliced and fused by channel to generate new first-layer, second-layer and third-layer feature maps, and the number of channels is changed based on the 1×1 convolution operation for subsequent processing.
[0072] Furthermore, in order to improve the lightness of the backbone network, the convolution layer of the YOLOv8 backbone network can be improved, and the standard convolution in the network can be replaced by the depthwise separable convolution, in which the depthwise convolution is applied to each input channel independently, and the point-by-point convolution is responsible for cross-channel information integration; the SE module in the EfficientNet network is used to model the interdependence between channels to enhance the feature expression capability, and the fully connected layer in the module is replaced by a 1×1 convolution layer, which also realizes cross-channel information aggregation, but saves the number of parameters and computation.
[0073] In this embodiment, the backbone network outputs multi-scale feature maps with different resolutions and channel numbers, which are passed to the neck network, such as Figure 2 and Figure 3 As shown in the figure, in order to enhance the model's attention to the area of interest, a layer of EMA attention mechanism and Ghost module are added to the connection module SPPF between the end of the YOLOv8 backbone network and the neck network; the EMA attention mechanism reshapes some feature channels to achieve batch dimensions, dynamically adjusts the channel weights of the parallel sub-networks through global information encoding, and promotes cross-dimensional interaction between the output features of the two sub-networks, thereby effectively integrating information, avoiding dimensionality reduction and the increase of computational complexity, and improving model performance.
[0074] Optionally, the training set is input into the backbone network of the improved twin EfficientNet-YOLOv8, where the feature maps of the same size generated by the EfficientNet network and the YOLOv8 network in each layer module are represented as follows:
[0075] (p1 EfficientNet 、p2 EfficientNet 、p3 EfficientNetf )、(p1 yolov8 、p2 yolov8 、p3 yolov8 );
[0076] The fusion features of the same layer are represented as (p1, p2, p3), and the feature processing method is expressed as:
[0077] p1=conv(1×1)(concatenate(p1 EfficientNet ,p1 yolov8 )); where concatenate() is the channel concatenation function;
[0078] Specifically, the processing of fusion features is as follows: Each fusion feature is expressed as: X∈R H×W×C , and divided into T cross-channel sub-features, the input features of each fusion feature are: X = [X0, X1, ... X t ,...,X T-1 ], the result of feature pooling is:
[0079] Where: H, W, C represent the length, width, and channel dimensions of each feature map, respectively. Respectively represent the feature pooling processing results of horizontal width and vertical height dimensions.
[0080] Among them, in order to reduce the large amount of computation caused by a large number of convolution kernels in the feature map, the basic feature map is generated by using conventional convolution in the Ghost module, and then the Ghost feature map is expanded based on low-computation linear transformation. These feature maps can efficiently reproduce and enrich the feature information extracted by the initial convolution layer and achieve performance improvement with lower computational complexity.
[0081] Optionally, the above method further includes:
[0082] S21, based on the feature pooling processing result, a basic feature map is generated using conventional convolution in the Ghost module, and is expanded into a Ghost feature map based on a linear transformation with low computational complexity, and the Ghost feature map is determined as a feature image. The specific calculation method is:
[0083] Y = X × f + b, where Y is the basic feature map obtained by conventional convolution operation; f is the convolution filter; b is the bias term, and X represents the fused feature after feature pooling processing;
[0084] Y ij =ψ i,j (Y i ), where Y ij is the Ghost feature map after linear transformation expansion, ψ i,j is a linear processing function, Y i represents the i-th feature map in the basic feature map, and j is the j-th linear operation performed on the basic feature map.
[0085] S3, fuses and adjusts the feature images of different scales and generates feature representations. The target prediction results in each sample image are obtained through the output layer prediction of the detection model. The target prediction results include target position, target size and confidence score.
[0086] Optionally, the above-mentioned feature images of different scales are fused and adjusted to generate feature representations, specifically:
[0087] Utilizing the bottom-up and top-down structures, feature images of different scales are fused and adjusted based on the feature pyramid network, and the corresponding feature representations are obtained.
[0088] Among them, BiFPN introduces bidirectional connections between adjacent levels of the feature pyramid based on the feature pyramid network that fuses feature maps from different stages of the backbone network, using bottom-up and top-down structures. First, by combining upsampling and element-by-element addition, higher-level semantic information is passed to lower-level features (top-down path). However, for the detection of small targets, relying solely on the top-down path may not be enough, because small target features are easily lost by pooling in the early stages. Therefore, a bottom-up path is introduced to pass feature information along a specific path, pass on strong positioning features from the lower layers, and ensure that the detailed features of small targets can be fully retained, thereby achieving effective aggregation of high- and low-level feature information.
[0089] Optionally, the target prediction result in each sample image is obtained by predicting the output layer of the detection model, specifically:
[0090] Information is extracted from each feature representation through two 3×3 convolutions and one 1×1 convolution, and the original prediction value is output as the center coordinates, width and height of the bounding box, as well as the category confidence score to obtain the target prediction result.
[0091] The detection head uses a decoupled head structure to separate the classification and detection heads, generate anchor frames of different sizes, and are responsible for predicting the target frame and category probability at that scale, and offset and resize the anchor frame to match the real target. On the fused multi-scale feature map, two 3×3 convolutions and one 1×1 convolution are used to extract information, and the original prediction values are output as the center coordinates, width and height of the bounding box, as well as the category confidence score.
[0092] S4, based on the improved loss function, uses the validation set to evaluate the prediction performance of the detection model, and uses each evaluation result and loss function value to adjust the detection model until the detection model reaches the end of training conditions.
[0093] Among them, the total loss of the YOLOv8 model is the weighted sum of the positioning loss and the classification loss. The positioning loss measures the difference between the predicted target box and the true annotation box, while the classification loss evaluates the accuracy of the model's prediction of the target category and is trained according to the loss function value. During the training process, the model backpropagates from the output layer to the input layer based on the total loss function, uses the gradient descent method to update the network weights, determines the contribution of each layer parameter to the total loss, and minimizes the loss to make the predicted target box closer to the actual annotation box, thereby improving the accuracy of target positioning and classification.
[0094] Optionally, the above loss function is specifically:
[0095] L total =αL cls +βL DFL +δL CIOU +(1-α-β-δ)L sia ,in:
[0096]
[0097] Where, L total Represents the loss function value, L cls is the classification loss, L DFL is the classification focal loss, L CIOU is the complete border loss, α, β, δ are weight coefficients, and α, β, δ∈[0,1], L sia is the matching loss of the detection model, N represents the number of training samples in the training set, m is the distance threshold of dissimilar samples, Y is the label of whether two samples match, Y = 1 means the two samples are similar or matched, Y = 0 means the two samples do not match, E w Represents the similarity value between two training samples.
[0098] S5, preprocessing the image to be tested taken by the drone, and inputting the preprocessed image to be tested into the detection model that meets the training end condition for processing, so as to obtain the target detection result of the image to be tested.
[0099] Embodiment 2: The present application provides a target detection system based on an improved twin network, which is applied to any target detection method based on an improved twin network in Embodiment 1, such as Figure 4 As shown, including:
[0100] The first module is used to preprocess several sample images taken by the drone, and construct a data set for model training through the preprocessed sample images. The data set includes a training set, a validation set, and a test set.
[0101] The second module is used to input the training set into the backbone network of the detection model of the improved twin EfficientNet-YOLOv8 network, and extract feature images of different scales from several sample images of the training set through the introduced EMA mechanism and Ghost module.
[0102] The third module is used to fuse and adjust the feature images of different scales and generate feature representations. The target prediction results in each sample image are obtained through the output layer prediction of the detection model. The target prediction results include target position, target size and confidence score.
[0103] The fourth module is used to evaluate the prediction performance of the detection model using the validation set based on the improved loss function, and adjust the detection model using each evaluation result and loss function value until the detection model reaches the end of training conditions.
[0104] The fifth module is used to preprocess the test images taken by the drone, and input the preprocessed test images into the detection model that meets the training end conditions for processing to obtain the target detection results of the test images.
[0105] Embodiment 3: This embodiment of the present application provides an electronic device, such as Figure 5 As shown, it includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any method in Embodiment 1 is implemented.
[0106] Embodiment 4: The embodiment of the present application provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions enable a computer to execute any method in Embodiment 1.
[0107] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. The target detection method based on the improved twin network is characterized by: The specific steps include: Preprocessing a number of sample images taken by the drone, and constructing a data set for model training through the preprocessed sample images, wherein the data set includes a training set, a validation set, and a test set; The training set is input into the backbone network of the detection model of the improved twin EfficientNet-YOLOv8 network, and feature images of different scales are extracted from several sample images of the training set by introducing the EMA mechanism and the Ghost module; Each feature image of different scales is fused and adjusted to generate a feature representation, and the target prediction result in each sample image is obtained through the output layer prediction of the detection model, wherein the target prediction result includes the target position, target size and confidence score; Based on the improved loss function, the prediction performance of the detection model is evaluated using the validation set, and the detection model is adjusted using each evaluation result and the loss function value until the detection model reaches the training end condition; The images to be tested taken by the drone are preprocessed, and the preprocessed images to be tested are input into the detection model that meets the training end conditions for processing to obtain the target detection results of the images to be tested.
2. The target detection method based on the improved twin network according to claim 1, characterized in that: The preprocessing of the sample images taken by the drone is specifically as follows: Plan the flight path and shooting strategy of the drone according to the distribution of power lines and inspection requirements, take target images of insulators in different time periods and weather conditions, and preliminarily screen target images that meet the preset conditions; Adjusting image parameters of a target image that meets preset conditions, transforming the target image after the image parameters are adjusted, and determining a sample image from the transformed target image and the target image after the image parameters are adjusted; The detection targets in the sample images of uniform size are annotated to obtain a data set, and the data set is divided according to a preset ratio to obtain a training set, a validation set and a test set included in the data set.
3. The target detection method based on the improved twin network according to claim 1, characterized in that: In the improved twin EfficientNet-YOLOv8 network: the standard convolution of the convolution layer in the YOLOv8 network is replaced with a depthwise separable convolution, wherein: the depthwise separable convolution is applied to each input channel, and the pointwise convolution in the YOLOv8 network integrates information across channels; the fully connected layer of the SE module in the EfficientNet network is replaced with a 1×1 convolution layer for cross-channel information aggregation; a layer of EMA attention mechanism and Ghost module are added to the connection module SPPF between the end of the YOLOv8 network and the neck network, wherein: The training set is input into the backbone network of the improved twin EfficientNet-YOLOv8, where the feature maps of the same size generated by the EfficientNet network and the YOLOv8 network in each layer module are represented as follows: (p1 EfficientNet 、p2 EfficientNet 、p3 EfficientNetf )、(p1 yolov8 、p2 yolov8 、p3 yolov8 ); The fusion features of the same layer are represented as (p1, p2, p3), and the feature processing method is expressed as: p1=conv(1×1)(concatenate(p1 EfficientNet ,p1 yolov8 )); where concatenate() is the channel concatenation function; The processing of fusion features is as follows: Each fusion feature is expressed as: X∈R H×W×C , and divided into T cross-channel sub-features, the input features of each fusion feature are: X = [X0, X1, ... X t ,...,X T-1 ], the result of feature pooling is: Where: H, W, C represent the length, width, and channel dimensions of each feature map, respectively. Respectively represent the feature pooling processing results of horizontal width and vertical height dimensions.
4. The target detection method based on the improved twin network according to claim 3 is characterized in that: The method further comprises: Based on the result of feature pooling, the basic feature map is generated by conventional convolution in the Ghost module, and is expanded into a Ghost feature map based on a linear transformation with low computational complexity, and the Ghost feature map is determined as a feature image. The specific calculation method is as follows: Y = X × f + b, where Y is the basic feature map obtained by conventional convolution operation; f is the convolution filter; b is the bias term, and X represents the fused feature after feature pooling processing; Y ij =ψ i,j (Y i ), where Y ij is the Ghost feature map after linear transformation expansion, ψ i,j is a linear processing function, Y i represents the i-th feature map in the basic feature map, and j is the j-th linear operation performed on the basic feature map.
5. The target detection method based on the improved twin network according to claim 1, characterized in that: The feature images of different scales are fused and adjusted to generate feature representations, specifically: By utilizing the bottom-up and top-down structures, feature images of different scales are fused and adjusted based on the feature pyramid network, and the corresponding feature representations are obtained.
6. The target detection method based on the improved twin network according to claim 1, characterized in that: The target prediction results in each sample image are obtained by predicting the output layer of the detection model, specifically: Information is extracted from each feature representation through two 3×3 convolutions and one 1×1 convolution, and the original prediction value is output as the center coordinates, width and height of the bounding box, as well as the category confidence score, to obtain the target prediction result.
7. The target detection method based on the improved twin network according to claim 1, characterized in that: The loss function is specifically: L total =αL cls +βL DFL +δL CIOU +(1-a-b-d)L sia , among which: Where, L total Represents the loss function value, L cls is the classification loss, L DFL is the classification focal loss, L CIOU is the complete border loss, α, β, δ are weight coefficients, and α, β, δ∈[0,1], L sia is the matching loss of the detection model, N represents the number of training samples in the training set, m is the distance threshold of dissimilar samples, Y is the label of whether two samples match, Y = 1 means the two samples are similar or matched, Y = 0 means the two samples do not match, E w Represents the similarity value between two training samples.
8. An improved twin network-based target detection system, applied to the improved twin network-based target detection method according to any one of claims 1 to 7, characterized in that: include: The first module is used to preprocess a number of sample images taken by the drone, and construct a data set for model training through the preprocessed sample images, wherein the data set includes a training set, a validation set, and a test set; The second module is used to input the training set into the backbone network of the detection model of the improved twin EfficientNet-YOLOv8 network, and extract feature images of different scales from several sample images of the training set through the introduced EMA mechanism and Ghost module; The third module is used to fuse and adjust the feature images of different scales and generate feature representations, and obtain the target prediction results in each sample image through the output layer prediction of the detection model, wherein the target prediction results include the target position, target size and confidence score; The fourth module is used to evaluate the prediction performance of the detection model using the validation set based on the improved loss function, and adjust the detection model using each evaluation result and the loss function value until the detection model reaches the training end condition; The fifth module is used to preprocess the test images taken by the drone, and input the preprocessed test images into the detection model that meets the training end conditions for processing to obtain the target detection results of the test images.
9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the method according to any one of claims 1 to 7 is implemented when the processor executes the computer program.
10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, which enable a computer to execute the method of any one of claims 1-7.