Deep network target detection method based on multi-scale refined feature fusion

Abstract

The embodiment of the invention provides a deep network target detection method based on multi-scale refined feature fusion, and belongs to the technical field of multi-target feature detection. Comprising the following steps: acquiring inspection images and target labels of a plurality of power transmission lines, and forming a training sample set; according to the method, the DBNet network model is firstly adopted to perform feature extraction so as to perform preliminary prediction on a target in the inspection image, then the features extracted from the DBNet network model are input into the YOLO Head network model for further prediction, and finally the optimal solution and the corresponding model parameters are obtained by combining the loss functions of the probability graph, the threshold graph, the binary graph and the anchor graph. Training a DBNet network model and a YOLO Head network model according to the training result; according to the mode, different electric power transmission line targets can be accurately detected and identified, and the detection precision and the identification accuracy are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-target feature detection, and in particular to a deep network target detection method based on multi-scale refined feature fusion. Background Art

[0002] Drone inspection is a way to inspect and check power lines, which can be achieved by using drones equipped with different sensors or devices. During the drone inspection process, drone pilots and power line inspectors need to work closely together to ensure that the drone flies along the correct route and direction, while taking photos or videos of the power lines and transferring the data to a computer or mobile device for analysis and processing. Therefore, deep learning-based drone power line inspection target defect detection in power production environments is becoming increasingly important.

[0003] At present, the traditional target defect detection method uses a fixed window for sliding. When the object is small, the window will frame a lot of background; when the target is large, the serial port may frame part of the object, which will lead to misclassification of the target. Therefore, this method can usually only have a good effect in specific scenes. When the environment such as lighting changes or the object scale is different, the effect may be greatly reduced, and there is a problem of low recognition accuracy.

[0004] The inventors of the present application discovered during the process of implementing the present invention that the above-mentioned solution in the prior art has the defect of low recognition accuracy. Summary of the invention

[0005] The purpose of an embodiment of the present invention is to provide a deep network target detection method based on multi-scale refined feature fusion, which has a function of high recognition accuracy.

[0006] In order to achieve the above object, an embodiment of the present invention provides a deep network target detection method based on multi-scale refined feature fusion, including:

[0007] Obtain inspection images and target labels of multiple power transmission lines and form a training sample set;

[0008] Construct DBNet network model;

[0009] Inputting a plurality of the inspection images into the DBNet network model to obtain a feature map, a probability map, a threshold map, and a binary map of the inspection images;

[0010] Build the YOLO Head network model;

[0011] Input the feature map into the YOLO Head network model to obtain an anchor map and a predicted value;

[0012] Obtain a total loss function according to the probability map, the threshold map, the binary map, and the anchor map;

[0013] Training the DBNet network model and the YOLO Head network model according to the optimal solution of the loss function;

[0014] Acquiring a real-time inspection image of the power transmission line;

[0015] The real-time inspection image is input into the DBNet network model to perform multi-target inspection.

[0016] Optionally, inputting a plurality of the inspection images into the DBNet network model to obtain a feature map, a probability map, a threshold map and a binary map of the inspection images includes:

[0017] The Backbone network in the DBNet network model performs convolution calculation on the inspection image to obtain an image of 1 / 2 size;

[0018] Downsampling the 1 / 2 size image four times in sequence to obtain first feature maps of four scales, wherein the sizes of the four first feature maps include 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the inspection image;

[0019] The FPN network in the DBNet network model performs convolution and upsampling operations on the four first feature maps respectively to obtain four second feature maps;

[0020] The four second feature maps are fused and spliced ​​to obtain an output feature map.

[0021] Optionally, inputting the plurality of inspection images into the DBNet network model to obtain a feature map, a probability map, a threshold map and a binary map of the inspection images further comprises:

[0022] Mapping the output feature map to the original image size, and merging the generated maps to obtain a probability map and a threshold map;

[0023] A binary image is obtained according to the probability image and the threshold image.

[0024] Optionally, acquiring a binary image according to the probability image and the threshold image includes:

[0025] According to formula (1), the binary image is obtained.

[0026]

[0027] in, is the binary image, P i,j is the probability map, T i,j is the adaptive threshold map, k is a hyperparameter, i and j are the x-axis and y-axis coordinates respectively.

[0028] Optionally, inputting the feature map into the YOLO Head network model to obtain an anchor map and a predicted value includes:

[0029] Obtain the last three of the four second feature maps;

[0030] Obtain corresponding anchor maps according to the last three second feature maps respectively;

[0031] Get the distance between each rectangular box and the true value box in the anchor image;

[0032] Acquire pixel values ​​of the target area according to the binary image and the inspection image;

[0033] Using the pixel value of the target area as a special category for target detection of the YOLO Head network model;

[0034] The prediction value is output by combining the special category and the distance between each rectangular box in the anchor image and the true value box.

[0035] Optionally, inputting the feature map into the YOLO Head network model to obtain an anchor map and a predicted value further includes:

[0036] Predict the box detection frame of each target based on the anchor map;

[0037] Get the area of ​​the box detection box;

[0038] Scaling the box detection frame to the size of the binary image;

[0039] Obtain pixel values ​​of the area corresponding to the box detection frame and the binary image;

[0040] According to formula (2), the pixel value ratio of the target area in the box detection frame is calculated.

[0041]

[0042] Among them, p is the pixel value ratio of the target area in the box detection frame, N target is the pixel value of the area corresponding to the box detection frame and the binary image, N box is the area of ​​the box detection box;

[0043] Determine whether the pixel value ratio is greater than or equal to a preset threshold;

[0044] When it is determined that the pixel value ratio is greater than or equal to a preset threshold, the cls category of the box detection box is set to the target category;

[0045] The confidence level of the box detection frame is adjusted to the pixel value ratio of the target area.

[0046] Optionally, obtaining a total loss function according to the probability map, the threshold map, the binary map, and the anchor map includes:

[0047] The loss function of the probability map and the binary map is calculated according to formula (3):

[0048]

[0049] Among them, LOSS gai is the probability graph loss function, LOSS er is the binary graph loss function, S l is the training sample set, x i is the x-axis coordinate of the ith sample, y i is the y-axis coordinate of the i-th sample, where i is an integer number.

[0050] Optionally, obtaining a total loss function according to the probability map, the threshold map, the binary map, and the anchor map further includes:

[0051] The threshold map loss function is calculated according to formula (4):

[0052]

[0053] Among them, LOSS yu is the threshold map loss function, R d is all the pixels in the area obtained after the annotation box is expanded by the D offset. The value of the i-th point in the threshold map calculated for the true label of the training sample set, is the value of the i-th point in the threshold map.

[0054] Optionally, obtaining a total loss function according to the probability map, the threshold map, the binary map, and the anchor map further includes:

[0055] Obtaining the classification loss function and coordinate loss function of the anchor graph;

[0056] The confidence loss function of the anchor graph is calculated according to formula (5):

[0057]

[0058] Among them, LOSS obj is the confidence loss function, N is the number of samples, n≤N, l i is the loss function of the i-th sample;

[0059] The total loss function is calculated according to formula (6):

[0060] LOSS=LOSS gai +α×LOSS yu +β×LOSS er +γ×LOSS box +δ×LOSS cls +ε×LOSS obj , (6)

[0061] Among them, LOSS is the total loss function, LOSS box is the coordinate loss function, LOSS cls is the classification loss function, α is the weight coefficient of the threshold image loss function, β is the weight coefficient of the binary image loss function, γ is the weight coefficient of the coordinate loss function, δ is the weight coefficient of the classification loss function, and ε is the weight coefficient of the confidence loss function.

[0062] On the other hand, the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, wherein the instructions are used to be read by a machine so that the machine executes any of the above-described deep object detection methods.

[0063] Through the above technical scheme, the deep network target detection method based on multi-scale refined feature fusion provided by the present invention first uses the DBNet network model to extract features to make a preliminary prediction of the targets in the inspection image, and then inputs the features extracted from the DBNet network model into the YOLO Head network model for further prediction. Finally, the optimal solution and the corresponding model parameters are obtained by combining the loss functions of the probability map, threshold map, binary map and anchor map to train the DBNet network model and the YOLO Head network model. This method can accurately detect and identify different power transmission line targets, effectively improving the detection accuracy and recognition accuracy.

[0064] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0066] Figure 1 is a flow chart of a deep network target detection method based on multi-scale refined feature fusion according to an embodiment of the present invention;

[0067] Figure 2 It is a flowchart of obtaining a binary image in a deep network target detection method based on multi-scale refined feature fusion according to an embodiment of the present invention;

[0068] Figure 3 It is a flowchart of obtaining prediction values ​​in a deep network target detection method based on multi-scale refined feature fusion according to an embodiment of the present invention;

[0069] Figure 4 It is a flowchart of obtaining a total loss function in a deep network target detection method based on multi-scale refined feature fusion according to an embodiment of the present invention;

[0070] Figure 5 It is a schematic diagram of feature downsampling of a DBNet network model in a deep network target detection method based on multi-scale refined feature fusion according to an embodiment of the present invention;

[0071] Figure 6 is a schematic diagram of feature fusion and splicing in a deep network target detection method based on multi-scale refined feature fusion according to an embodiment of the present invention;

[0072] Figure 7 It is a schematic diagram of generating a binary image in a deep network target detection method based on multi-scale refined feature fusion according to an embodiment of the present invention;

[0073] Figure 8 It is a schematic diagram of generating an anchor graph by a YOLO Head network model in a deep network target detection method based on multi-scale refined feature fusion according to an embodiment of the present invention;

[0074] Fig. 9 It is a schematic diagram of target detection and recognition of inspection images in a deep network target detection method based on multi-scale refined feature fusion according to an embodiment of the present invention. DETAILED DESCRIPTION

[0075] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0076] Figure 1 is a flowchart of a deep network target detection method based on multi-scale refined feature fusion according to an embodiment of the present invention. Figure 1 In the present invention, the deep network target detection method may include:

[0077] In step S10, inspection images and target labels of multiple power transmission lines are obtained to form a training sample set, wherein the target label is also a target classification, and specifically, the target classification may include equipment defect classification, and the inspection image and its corresponding target label may be used as a training sample.

[0078] In step S11, a DBNet network model is constructed.

[0079] In step S12, a plurality of inspection images are input into the DBNet network model to obtain feature maps, probability maps, threshold maps, and binary maps of the inspection images.

[0080] In step S13, a YOLO Head network model is constructed.

[0081] In step S14, the feature map is input into the YOLO Head network model to obtain the anchor map and the predicted value.

[0082] In step S15, a total loss function is obtained according to the probability map, the threshold map, the binary map and the anchor map. A more accurate loss function can be obtained by comprehensively considering the loss functions of the probability map, the threshold map, the binary map and the anchor map.

[0083] In step S16, the DBNet network model and the YOLO Head network model are trained according to the optimal solution of the total loss function. Among them, the optimal model parameters can be obtained according to the optimal solution of the total loss function, and the DBNet network model and the YOLO Head network model can be corrected according to the optimal model parameters. Specifically, during the training process, the output value of the network is first calculated by forward propagation, and then the output value is compared with the true label to obtain the value of the loss function. Then, the back propagation algorithm is used to calculate the gradient value of the loss function to the network parameters, that is, the contribution of each parameter in the loss function. Finally, the optimization algorithm such as gradient descent is used to update the network parameters according to the gradient value, so that the value of the loss function is continuously reduced, so that the network can better complete the training task. The core of the back propagation algorithm is the chain rule, that is, the gradient value is passed from the output layer to the input layer in sequence, and each layer of nodes can calculate its own gradient value according to the gradient value of the previous layer and its own activation function, and then the gradient value is passed to the previous layer of nodes until it is passed to the input layer, and finally the gradient value of each parameter is obtained. Back propagation of loss values ​​is an important step in neural network training. By calculating gradient values ​​through back propagation, the parameters of the DBNet network and the YOLO deep neural network can be continuously updated, making the network's prediction results more accurate.

[0084] In step S17, a real-time inspection image of the power transmission line is obtained, wherein a drone can be used to take real-time images on the power transmission line and transmit them to the backend.

[0085] In step S18, the real-time inspection image is input into the DBNet network model for multi-target inspection. The real-time inspection image is used as input, and the trained model can perform high-precision and efficient prediction, thereby obtaining the target defects of the inspection image.

[0086] In step S10 to step S18, first obtain inspection images of multiple power transmission lines and corresponding target labels, and summarize them as a training sample set. The inspection image is first input into the DBNet network model, its features are extracted, and the corresponding probability map, threshold map and binary map are obtained. The feature map extracted from the DBNet network model is then input into the YOLO Head network model to obtain multiple anchor maps, and then the rectangular boxes in the anchor map are further screened to obtain the predicted value closest to the true value. Finally, the total loss function can be obtained by combining the loss functions of the probability map, threshold map, binary map and anchor map. The optimal model parameters are obtained according to the optimal solution of the total loss function, and the DBNet network model parameters and the YOLO Head network model parameters are corrected using the optimal model parameters, thereby realizing the training of the DBNet network model and the YOLO Head network model. The real-time inspection image is input into the DBNet network model to obtain the target classification / target defect classification of multiple targets in the real-time inspection image. Specifically, it can be as follows Fig. 9 shown.

[0087] The traditional target defect detection method uses a fixed window for sliding. When the object is small, the window will frame a lot of background; when the target is large, the serial port may frame part of the object, which will lead to misclassification of the target. Therefore, this method can usually only have a relatively good effect in specific scenarios. When the environment such as lighting changes or the object scale is different, the effect may be greatly reduced, and there are problems such as high computing power requirements and low recognition accuracy. Specifically, if the target detection of the mainstream image global features is adopted, although the target recognition and classification can be achieved, the processing processes such as pre-processing, feature extraction and classification are relatively complicated due to the lack of background suppression, resulting in high computational requirements. In addition, the need to pre-set prior knowledge and select appropriate extraction methods according to task requirements will limit its ability to adapt to different tasks, and it will take a lot of time and manpower to design and optimize features, resulting in low recognition accuracy. In this embodiment of the present invention, the loss function of multi-label classification is adopted, which can accurately detect and identify different power transmission line targets, effectively improve the detection accuracy and recognition accuracy, and require low computing power.

[0088] In this embodiment of the present invention, when it is necessary to obtain a binary image of the inspection image, it is necessary to first extract the features of the inspection image. The specific steps can be as follows: Figure 2 Specifically, Figure 2 In the method, the deep network target detection method may further include:

[0089] In step S120, the Backbone network in the DBNet network model performs convolution calculation on the inspection image to obtain an image of 1 / 2 size. The Backbone network is mainly responsible for image extraction, using ResNet50 and ResNet18 networks. Specifically, when the inspection image [1, 3, 640, 640] is input, it enters the Backbone skeleton network and performs a convolution calculation first, with the size shifted to 1 / 2 of the original.

[0090] In step S121, the 1 / 2 size image is downsampled four times in sequence to obtain first feature maps of four scales, wherein the sizes of the four first feature maps include 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the inspection image. Specifically, Figure 5 shown.

[0091] In step S122, the FPN network in the DBNet network model performs convolution and upsampling operations on the four first feature maps to obtain four second feature maps. Among them, the feature pyramid structure FPN network is a convolutional network used to extract features of various dimensions in the inspection image. Specifically, the feature pyramid network FPN is used to fuse the different feature layers of ResNet50 and ResNet18 through bottom-up connections and lateral connections. The bottom-up lines generate P2, P3, P4, and P5 respectively.

[0092] In step S123, the four second feature maps are fused and spliced ​​to obtain an output feature map. The height and width of the feature map output after FPN calculation are 1 / 4 of the original image, that is, [1, 256, 160, 160]. Specifically, Figure 6 shown.

[0093] In step S124, the output feature map is mapped to the original image size, and the generated maps are merged to obtain a probability map and a threshold map. Among them, the Head network in the DBNet network model will upsample based on the output features of the FPN network, map the original 1 / 4 size of the FPN network output features to the original inspection image size, and finally generate three maps and merge them, and output [1, 3, 640, 640] to obtain a regional probability map and a regional threshold map.

[0094] In step S125, a binary image is obtained based on the probability image and the threshold image. Figure 7 As shown, a binary image can be obtained by performing a differentiable binarization DB operation on the threshold image and the probability image. Specifically, the steps may include:

[0095] According to formula (1), we can obtain a binary image.

[0096]

[0097] in, is a binary image, P i,j is the probability graph, T i,j is the adaptive threshold map, k is a hyperparameter, i and j are the x-axis coordinates and y-axis coordinates respectively. Specifically, DB is differentiable binarization, which solves the problem that it cannot be optimized with the training of the segmentation network. Therefore, a step function that approximates the binarization process is selected. The binary map, probability map, and adaptive threshold map have the same size, that is, formula (1) is the calculation formula for each point in the binary map.

[0098] In this embodiment of the present invention, in order to obtain the predicted value of the inspection image, the features extracted by the DBNet network model need to be input into the YOLO Head network model. The specific prediction steps can be as follows: Figure 3 Specifically, Figure 3 In the method, the deep network target detection method may further include:

[0099] In step S1400, the last three of the four second feature maps are obtained, wherein the last three of the second feature maps are P3, P4 and P5.

[0100] In step S1401, the corresponding anchor maps are obtained according to the last three second feature maps. Figure 8 As shown in the figure, the YOLO Head network uses three output branches to obtain P3, P4, and P5 to classify and regress boxes respectively, make identification, and generate multiple anchor maps.

[0101] In step S1402, the distance between each rectangular box in the anchor image and the true value box is obtained. The true value box is the box corresponding to the true value, and the distance between each rectangular box in the anchor image and the true value box is calculated respectively. The smaller the distance, the closer the predicted value is to the true value.

[0102] In step S1403, the pixel value of the target area is obtained according to the binary image and the inspection image. The pixel value of the target area can be obtained by performing pixel-level operations on the binary image and the inspection image, and the pixel values ​​of the non-target area are suppressed.

[0103] In step S1404, the pixel value of the target area is used as a special category for target detection of the YOLO Head network model. The pixel value of the target area is used as a special category to further filter out the predicted value that is closer to the true value. Specifically, the label of this category can be set to a larger value of 100.

[0104] In step S1405, the predicted value is output in combination with the special category and the distance between each rectangular box and the true value box in the anchor map. Among them, after the above two steps, the predicted value closest to the true value is immediately obtained and output. Specifically, the YOLOHead network model uses three feature maps of different scales, corresponding to objects of different sizes. Feature maps of different scales have different spatial resolutions and semantic information, and can effectively process objects of different sizes. Specifically, in the prediction process, the YOLO Head network predicts the feature maps of each scale separately. For each feature map, the YOLO Head network uses a convolutional layer to generate prediction results. The prediction results include the probability of the object category, the position of the object, and the confidence of the object. Among them, the predicted values ​​of the position and confidence of the object are based on the coordinates of the feature map, and the coordinates in the original image need to be obtained by inverse calculation. Finally, the YOLO Head network merges the prediction results of the three scales, and obtains the final detection result through threshold screening and non-maximum suppression (NMS).

[0105] In step S1406, a box detection frame of each target is predicted according to the anchor graph. The YOLO Head network model predicts a box detection frame of each target according to the anchor graph.

[0106] In step S1407, the area of ​​the box detection frame is obtained.

[0107] In step S1408, the box detection frame is scaled to the size of the binary image.

[0108] In step S1409, the pixel values ​​of the area corresponding to the box detection frame and the binary image are obtained, where the area is the overlap area of ​​the box detection frame and the binary image, and the pixel values ​​in the area are taken as the total number of pixels in the target area.

[0109] In step S1410, the pixel value ratio of the target area in the box detection frame is calculated according to formula (2):

[0110]

[0111] Among them, p is the pixel value ratio of the target area in the box detection box, N target is the pixel value of the area corresponding to the box detection box and the binary image, N box is the area of ​​the box detection box.

[0112] In step S1411, it is determined whether the pixel value ratio is greater than or equal to a preset threshold.

[0113] In step S1412, when it is determined that the pixel value ratio is greater than or equal to the preset threshold, the cls category of the box detection frame is set as the target category.

[0114] In step S1413, the confidence of the box detection frame is adjusted to the pixel value ratio of the target area. The above judgment can be shown as formula (7) and formula (8):

[0115]

[0116]

[0117] Among them, cls new is the category of the adjusted box detection box, conf new is the confidence of the adjusted box detection box, target is the target category, cls is the category of the original box detection box, conf is the confidence of the original box detection box, and p thres Specifically, this method can update the target category and confidence level according to the proportion of pixel values. The higher the confidence level, the closer the predicted value is to the true value, which further improves the prediction accuracy.

[0118] In step S1401 to step S1413, a plurality of anchor images are first generated according to P3, P4 and P5 in the second feature map, and the rectangular boxes close to the true value box are screened out according to the distance between each rectangular box in the anchor image and the true value box. Then, the pixel values ​​obtained by performing pixel-level operations on the binary image and the inspection image are used as special categories of the YOLO Head network model for target detection, and the predicted value closest to the true value can be obtained and output by combining the DBNet network model and the YOLO Head network model. Furthermore, the box detection frame of each target is matched with the binary image, the pixel value ratio of the target area in the box detection frame is calculated, and compared with the preset threshold, so that the target category and confidence can be effectively updated. The higher the confidence, the closer the predicted value is to the true value, which further improves the prediction accuracy.

[0119] In this embodiment of the present invention, in order to obtain the most accurate prediction value, it is also necessary to calculate and combine the loss functions of the probability map, threshold map, binary map and anchor map. The specific steps can be as follows: Figure 4 Specifically, Figure 4 In the method for detecting a deep object, the method may further include:

[0120] In step S150, the loss function of the probability map and the binary map is calculated according to formula (3):

[0121]

[0122] Among them, LOSS gai is the probability graph loss function, LOSS er is the binary graph loss function, S l is the training sample set, x i is the x-axis coordinate of the ith sample, y i is the y-axis coordinate of the i-th sample, where i is an integer number.

[0123] In step S151, the threshold map loss function is calculated according to formula (4):

[0124]

[0125] Among them, LOSS yu is the threshold map loss function, R d is all the pixels in the area obtained after the annotation box is expanded by the D offset. The value of the i-th point in the threshold map calculated for the true label of the training sample set, is the value of the i-th point in the threshold map.

[0126] In step S152, the classification loss function and the coordinate loss function of the anchor graph are obtained, wherein the classification loss function uses the cross entropy loss function, and the coordinate loss function uses the mean square error loss function.

[0127] In step S153, the confidence loss function of the anchor graph is calculated according to formula (5):

[0128]

[0129] Among them, LOSS obj is the confidence loss function, N is the number of samples, n is the sample number, and n≤N, l i is the loss function of the ith sample. Specifically, one input sample corresponds to one classification output (1 or 0), and the BCE (binary cross-entropy) loss function for data D(x, y) containing N samples can be obtained according to formula (5). In addition, the loss function for the ith sample can include l i =-ω(y i ·log x i +(1-y i )*log(1-x i )), where ω is a hyperparameter.

[0130] In step S154, the total loss function is calculated according to formula (6):

[0131] LOSS=LOSS gai +α×LOSS yu +β×LOSS er +γ×LOSS box +δ×LOSS cls +ε×LOSS obj , (6)

[0132] Among them, LOSS is the total loss function, LOSS box is the coordinate loss function, LOSS cls is the classification loss function, α is the weight coefficient of the threshold image loss function, β is the weight coefficient of the binary image loss function, γ is the weight coefficient of the coordinate loss function, δ is the weight coefficient of the classification loss function, and ε is the weight coefficient of the confidence loss function.

[0133] On the other hand, the present invention also provides a computer-readable storage medium, which stores instructions, and the instructions are used to be read by a machine so that the machine executes any of the above deep object detection methods.

[0134] Through the above technical scheme, the deep network target detection method based on multi-scale refined feature fusion provided by the present invention first uses the DBNet network model to extract features to make a preliminary prediction of the targets in the inspection image, and then inputs the features extracted from the DBNet network model into the YOLO Head network model for further prediction. Finally, the optimal solution and the corresponding model parameters are obtained by combining the loss functions of the probability map, threshold map, binary map and anchor map to train the DBNet network model and the YOLO Head network model. This method can accurately detect and identify different power transmission line targets, effectively improving the detection accuracy and recognition accuracy.

[0135] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0136] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0137] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0139] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0140] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0141] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0142] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0143] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A deep network target detection method based on multi-scale refined feature fusion, characterized in that: include: Obtain inspection images and target labels of multiple power transmission lines and form a training sample set; Construct DBNet network model; Inputting a plurality of the inspection images into the DBNet network model to obtain a feature map, a probability map, a threshold map, and a binary map of the inspection images; Build the YOLO Head network model; Input the feature map into the YOLO Head network model to obtain an anchor map and a predicted value; Obtain a total loss function according to the probability map, the threshold map, the binary map, and the anchor map; Training the DBNet network model and the YOLO Head network model according to the optimal solution of the loss function; Acquiring a real-time inspection image of the power transmission line; The real-time inspection image is input into the DBNet network model to perform multi-target inspection.

2. The deep network target detection method according to claim 1, characterized in that: Inputting a plurality of the inspection images into the DBNet network model to obtain a feature map, a probability map, a threshold map and a binary map of the inspection images includes: The Backbone network in the DBNet network model performs convolution calculation on the inspection image to obtain an image of 1 / 2 size; Downsampling the 1 / 2 size image four times in sequence to obtain first feature maps of four scales, wherein the sizes of the four first feature maps include 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the inspection image; The FPN network in the DBNet network model performs convolution and upsampling operations on the four first feature maps respectively to obtain four second feature maps; The four second feature maps are fused and spliced ​​to obtain an output feature map.

3. The deep network target detection method according to claim 2, characterized in that: Inputting a plurality of the inspection images into the DBNet network model to obtain a feature map, a probability map, a threshold map, and a binary map of the inspection images also includes: Mapping the output feature map to the original image size, and merging the generated maps to obtain a probability map and a threshold map; A binary image is obtained according to the probability image and the threshold image.

4. The deep network target detection method according to claim 3 is characterized in that: Acquiring a binary image according to the probability image and the threshold image includes: According to formula (1), the binary image is obtained. in, is the binary image, P i,j is the probability map, T i,j is the adaptive threshold map, k is a hyperparameter, i and j are the x-axis and y-axis coordinates respectively.

5. The deep network target detection method according to claim 4, characterized in that: Inputting the feature map into the YOLO Head network model to obtain the anchor map and the predicted value includes: Obtain the last three of the four second feature maps; Obtain corresponding anchor maps according to the last three second feature maps respectively; Get the distance between each rectangular box and the true value box in the anchor image; Acquire pixel values ​​of the target area according to the binary image and the inspection image; Using the pixel value of the target area as a special category for target detection of the YOLO Head network model; The prediction value is output by combining the special category and the distance between each rectangular box in the anchor image and the true value box.

6. The deep network target detection method according to claim 5, characterized in that: Inputting the feature map into the YOLO Head network model to obtain the anchor map and the predicted value also includes: Predict the box detection frame of each target based on the anchor map; Get the area of ​​the box detection box; Scaling the box detection frame to the size of the binary image; Obtain pixel values ​​of the area corresponding to the box detection frame and the binary image; According to formula (2), the pixel value ratio of the target area in the box detection frame is calculated. Among them, p is the pixel value ratio of the target area in the box detection frame, N target is the pixel value of the area corresponding to the box detection frame and the binary image, N box is the area of ​​the box detection box; Determine whether the pixel value ratio is greater than or equal to a preset threshold; When it is determined that the pixel value ratio is greater than or equal to a preset threshold, the cls category of the box detection box is set to the target category; The confidence level of the box detection frame is adjusted to the pixel value ratio of the target area.

7. The deep object detection method according to claim 1, characterized in that: Acquiring a total loss function according to the probability map, the threshold map, the binary map, and the anchor map includes: The loss function of the probability map and the binary map is calculated according to formula (3): Among them, LOSS gai is the probability graph loss function, LOSS er is the binary graph loss function, S l is the training sample set, x i is the x-axis coordinate of the ith sample, y i is the y-axis coordinate of the i-th sample, where i is an integer number.

8. The deep object detection method according to claim 7, characterized in that: Acquiring a total loss function according to the probability map, the threshold map, the binary map, and the anchor map further includes: The threshold map loss function is calculated according to formula (4): Among them, LOSS yu is the threshold map loss function, R d is all the pixels in the area obtained after the annotation box is expanded by the D offset. The value of the i-th point in the threshold map calculated for the true label of the training sample set, is the value of the i-th point in the threshold map.

9. The deep object detection method according to claim 8, characterized in that: Acquiring a total loss function according to the probability map, the threshold map, the binary map, and the anchor map further includes: Obtaining the classification loss function and coordinate loss function of the anchor graph; The confidence loss function of the anchor graph is calculated according to formula (5): Among them, LOSS obj is the confidence loss function, N is the number of samples, n≤N, l i is the loss function of the i-th sample; The total loss function is calculated according to formula (6): LOSS=LOSS gai +α×LOSS yu +β×LOSS er +γ×LOSS box +δ×LOSS cls +ε×LOSS obj , (6) Wherein, LOSS is the total loss function, LOSS box is the coordinate loss function, LOSS cls is the classification loss function, α is the weight coefficient of the threshold image loss function, β is the weight coefficient of the binary image loss function, γ is the weight coefficient of the coordinate loss function, δ is the weight coefficient of the classification loss function, and ε is the weight coefficient of the confidence loss function.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and the instructions are used to be read by a machine so that the machine executes the deep object detection method according to any one of claims 1 to 9.

Images