Training method of target detection network model, target detection method and related equipment

Abstract

The present application discloses a training method for a target detection network model, a target detection method and related equipment, and relates to the field of computer technology. The method includes: obtaining a power grid image data set; performing amplification processing on the power grid image data set to obtain an amplified power grid image data set; determining multi-label information of each power grid image in the amplified power grid image data set and a first loss corresponding to the multi-label information, wherein the multi-label information includes labels of each non-defective device in the power grid image; constructing a target detection network model according to a region proposal network and an R-CNN head for predicting target categories and relative positions; determining a total loss function according to the losses of the region proposal network and the R-CNN head and the first loss; inputting the amplified power grid image data set into the target detection network model, and training the target detection network model using the total loss function until the total loss function converges to obtain a target detection network model. The present application can improve the accuracy and robustness of target detection.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a training method for a target detection network model, a target detection method, and related equipment. Background Art

[0002] In the power grid scenario, defective objects and non-defective objects in the power grid image often have similar structures and textures. Defective objects refer to devices that are incomplete in the power grid image. Correspondingly, non-defective objects refer to devices that are complete in the power grid image and whose categories can be clearly determined in the power grid image. However, in target detection in the power grid scenario, defective objects are usually regarded as foreground targets and non-defective images are regarded as background targets. Foreground targets refer to objects of interest in target detection, and background targets refer to objects that users are not interested in during the target detection process, so they are regarded as background. This is significantly different from general datasets, in which similar objects are either foreground targets or background targets. Therefore, when training power grid images, deep neural network models that perform well in general datasets regard non-defective objects as background targets. The network tends to focus on the differences between defective objects and non-defective objects and ignore the common features between defective objects and non-defective objects, thereby affecting the feature expression ability and generalization ability of the target detection network model. Summary of the invention

[0003] In view of the above problems, the present application provides a training method for a target detection network model, a target detection method and related equipment.

[0004] In order to solve the above technical problems, this application proposes the following solutions:

[0005] In a first aspect, the present application provides a method for training a target detection network model, the method comprising: obtaining a power grid image dataset; performing an amplification process on the power grid image dataset to obtain an amplified power grid image dataset; determining multi-label information of each power grid image in the amplified power grid image dataset and a first loss corresponding to the multi-label information, the multi-label information including the labels of each non-defective device in the power grid image; constructing a target detection network model based on a region proposal network and an R-CNN head for predicting target categories and relative positions; determining a total loss function based on the losses of the region proposal network and the R-CNN head and the first loss; inputting the amplified power grid image dataset into the target detection network model, and training the target detection network model using the total loss function until the total loss function converges to obtain the target detection network model.

[0006] In combination with the first aspect, in a possible implementation, defect-free devices in each power grid image in the power grid image dataset are annotated by a labeling box, wherein defect-free devices of the same category but different positions in the power grid image use the same labeling box.

[0007] In combination with the first aspect, in another possible implementation, multi-label information of each power grid image in the augmented power grid image dataset is determined according to the overlap between each power grid image in the augmented power grid image dataset and the annotation box of each power grid image in the power grid image dataset.

[0008] In combination with the first aspect, in another possible implementation, the target detection network model further includes a fully connected layer, which is used to score the power grid image according to the presence of non-defective devices in the power grid image; determining a first loss corresponding to the multi-label information includes: according to the formula: L M =XX*Labels+log1+exp-X determines the first loss, where L M represents the first loss, X represents the scoring result of the fully connected layer on the power grid image, and labels represents the multi-label information of the power grid image.

[0009] In combination with the first aspect, in another possible implementation, according to the formula: L 总 =L rpn_cls +L rpn_bbox +L rcnn_cls +L rcnn_bbox +r*L M Determine the total loss function, where L rpn_cls represents the classification loss in the region proposal network, L rpn_bbox represents the bounding box loss in the region proposal network, L rcnn_cls represents the classification loss in the R-CNN head, L rcnn_bbox represents the bounding box loss in the R-CNN head and r represents the weight of the first loss.

[0010] In a second aspect, the present application provides a target detection method, which includes: obtaining an image of a power grid to be detected; inputting the image of the power grid to be detected into a trained target detection network model to obtain a detection result of the image of the power grid to be detected.

[0011] In a third aspect, the present application provides a training device for a target detection network model, and the training device for a target detection network model includes:

[0012] An acquisition module, used for acquiring a power grid image dataset;

[0013] An amplification module, used for amplifying the power grid image data set to obtain an amplified power grid image data set;

[0014] A label module, used to determine multi-label information of each power grid image in the augmented power grid image data set and a first loss corresponding to the multi-label information, wherein the multi-label information includes a label of each non-defective device in the power grid image;

[0015] A building block for building an object detection network model based on a region proposal network and an R-CNN head for predicting object categories and relative positions;

[0016] A determination module for determining a total loss function based on the losses of the region proposal network and the R-CNN head and the first loss;

[0017] The training module is used to input the augmented power grid image data set into the target detection network model, and use the total loss function to train the target detection network model until the total loss function converges to obtain the target detection network model.

[0018] In combination with the third aspect, in a possible implementation, the labeling module is specifically used to label the defect-free devices in each power grid image in the power grid image dataset through a labeling box, wherein the defect-free devices of the same category but different positions in the power grid image use the same labeling box.

[0019] In combination with the third aspect, in another possible implementation method, the label module is specifically used to determine the multi-label information of each power grid image in the augmented power grid image dataset based on the overlap between each power grid image in the augmented power grid image dataset and the annotation box of each power grid image in the power grid image dataset.

[0020] In combination with the third aspect, in another possible implementation, the target detection network model further includes a fully connected layer, and the fully connected layer is used to score the power grid image according to the presence of non-defective devices in the power grid image. The label module is specifically used to: determine the first loss according to the formula: LM = XX*Labels+log1+exp-X, where L M represents the first loss, X represents the scoring result of the fully connected layer on the power grid image, and labels represents the multi-label information of the power grid image.

[0021] In conjunction with the third aspect, in another possible implementation, the determining module is specifically configured to determine the value of L according to the formula: 总 =L rpn_cls +L rpn_bbox +L rcnn_cls L rcnn_bbox +r*L M Determine the total loss function, where L rpn_cls represents the classification loss in the region proposal network, L rpn_bbox represents the bounding box loss in the region proposal network, L rcnn_cls represents the classification loss in the R-CNN head, L rcnn_bboxrepresents the bounding box loss in the R-CNN head and r represents the weight of the first loss.

[0022] In a fourth aspect, the present application provides a target detection device, the device comprising:

[0023] An acquisition module, used for acquiring an image of a power grid to be detected;

[0024] The detection module is used to input the power grid image to be detected into the trained target detection network model to obtain the detection result of the power grid image to be detected.

[0025] In order to achieve the above-mentioned purpose, according to the fifth aspect of the present application, a storage medium is provided, which includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the training method of the target detection network model of the first aspect or the target detection method of the second aspect.

[0026] In order to achieve the above-mentioned purpose, according to the sixth aspect of the present application, an electronic device is provided, which includes at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call the program instructions in the memory to execute the training method of the target detection network model of the first aspect or the target detection method of the second aspect.

[0027] By means of the above technical solution, the technical solution provided by this application has at least the following advantages:

[0028] The present application provides a training method for a target detection network model, a target detection method and related equipment. The present application improves the performance of the backbone network by adding multi-label classification without directly interfering with the region proposal network RPN and the R-CNN head for predicting the target category and relative position. The feature alignment of defective images and non-defective images improves the robustness of the model in the face of complex backgrounds. Secondly, the classification head added to the target detection network model is not coupled with the backbone, which means that the classifier can work well with any other two-stage target detection network or any other backbone network.

[0029] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0031] Figure 1 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown;

[0032] Figure 2 A schematic diagram of a process for training a target detection network model provided in an embodiment of the present application is shown;

[0033] Figure 3 A schematic diagram of a power grid image provided by an embodiment of the present application is shown;

[0034] Figure 4 A schematic diagram of the structure of a target detection network model provided in an embodiment of the present application is shown;

[0035] Figure 5 A schematic diagram of a process flow of a target detection method provided in an embodiment of the present application is shown;

[0036] Figure 6 A schematic diagram of the structure of a training device for a target detection network model provided in an embodiment of the present application is shown;

[0037] Figure 7 A schematic structural diagram of a target detection device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0038] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0039] In the embodiments of the present application, the terms "first" and "second" do not have a logical or temporal dependency, nor do they limit the quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another element.

[0040] In the embodiments of the present application, the term "at least one" means one or more, and the term "plurality" means two or more.

[0041] It should also be understood that the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting." Similarly, the phrase "if it is determined that ..." or "if [a stated condition or event] is detected" may be interpreted as "upon determining that ..." or "in response to determining that ..." or "upon detecting [a stated condition or event]" or "in response to detecting [a stated condition or event]," depending on the context.

[0042] As described in the background technology, in the power grid scenario, defective objects and non-defective objects in the power grid image often have similar structures and textures. Defective objects refer to devices that are incomplete in the power grid image. Correspondingly, non-defective objects refer to devices that are complete in the power grid image and whose categories can be clearly determined in the power grid image. However, in target detection in the power grid scenario, defective objects are usually regarded as foreground targets and non-defective images are regarded as background targets. Foreground targets refer to objects of interest in target detection, and background targets refer to objects that users are not interested in during the target detection process, so they are regarded as background. This is significantly different from general data sets, in which similar objects are either foreground targets or background targets. Therefore, when training power grid images, deep neural network models that perform well in general data sets regard non-defective objects as background targets. The network tends to focus on the differences between defective objects and non-defective objects and ignore the common features between defective objects and non-defective objects, thereby affecting the feature expression ability and generalization ability of the target detection network model.

[0043] In view of this, an embodiment of the present application provides a training method for a target detection network model, and the specific method includes: obtaining a power grid image dataset; performing an amplification process on the power grid image dataset to obtain an amplified power grid image dataset; determining multi-label information of each power grid image in the amplified power grid image dataset and a first loss corresponding to the multi-label information, wherein the multi-label information includes the labels of each non-defective device in the power grid image; constructing a target detection network model based on a region proposal network and an R-CNN head for predicting target categories and relative positions; determining a total loss function based on the losses of the region proposal network and the R-CNN head and the first loss; inputting the amplified power grid image dataset into the target detection network model, and training the target detection network model using the total loss function until the total loss function converges to obtain a target detection network model.

[0044] In this application, adding multi-label classification can improve the performance of the backbone network without directly interfering with the region proposal network RPN and the R-CNN head used to predict the target category and relative position. The feature alignment of defect images and non-defect images improves the robustness of the model in the face of complex backgrounds. Secondly, the classification head added to the target detection network model is not coupled with the backbone, which means that the classifier can work well with any other two-stage target detection network or any other backbone network.

[0045] The embodiment of the present application also provides a training device for a target detection network model, which can be used to execute the training method for the target detection network model. Optionally, the training device for the target detection network model can be an electronic device with data processing capabilities, or a functional module in the electronic device, which is not limited to this.

[0046] For example, the electronic device may be a server, which may be a single server, or a server cluster consisting of multiple servers. For another example, the electronic device may be a mobile phone, a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, and a cellular phone, a personal digital assistant (PDA), an augmented reality (AR), a virtual reality (VR) device and other terminal devices. For another example, the electronic device may also be a recording device, a video surveillance device and other devices. The present application does not impose any special restrictions on the specific form of the electronic device.

[0047] The following example takes the training device of the target detection network model as an electronic device. Figure 1 As shown, Figure 1 The present application provides a hardware structure of an electronic device 100.

[0048] like Figure 1 As shown, the electronic device 100 includes a processor 110 , a communication circuit 120 and a communication interface 130 .

[0049] Optionally, the electronic device 100 may further include a memory 140 . The processor 110 , the memory 140 and the communication interface 130 may be connected via a communication line 120 .

[0050] The processor 110 may be a central processing unit (CPU), a general-purpose processor, a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof. The processor 110 may also be any other device having a processing function, such as a circuit, a device, or a software module, without limitation.

[0051] In one example, the processor 110 may include one or more CPUs, such as Figure 1 CPU0 and CPU1 in.

[0052] As an optional implementation, the electronic device 100 includes multiple processors, for example, in addition to the processor 110, it may also include a processor 170. The communication line 120 is used to transmit information between the components included in the electronic device 100.

[0053] The communication interface 130 is used to communicate with other devices or other communication networks. The other communication networks may be Ethernet, Radio Access Network (RAN), Wireless Local Area Networks (WLAN), etc. The communication interface 130 may be a module, a circuit, a transceiver or any device capable of achieving communication.

[0054] The memory 140 is used to store instructions, where the instructions may be computer programs.

[0055] Among them, the memory 140 can be a read-only memory (ROM) or other types of static storage devices that can store static information and / or instructions, or a random access memory (RAM) or other types of dynamic storage devices that can store information and / or instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), magnetic disk storage medium or other magnetic storage devices, etc., without limitation.

[0056] It should be noted that the memory 140 may exist independently of the processor 110 or may be integrated with the processor 110. The memory 140 may be used to store instructions or program codes or some data, etc. The memory 140 may be located inside the electronic device 100 or outside the electronic device 100, without limitation.

[0057] The processor 110 is used to execute the instructions stored in the memory 140 to implement the communication method provided in the following embodiments of the present application. For example, when the electronic device 100 is a terminal or a chip in a terminal, the processor 110 can execute the instructions stored in the memory 140 to implement the steps performed by the transmitting end in the following embodiments of the present application.

[0058] As an optional implementation, the electronic device 100 further includes an output device 150 and an input device 160. The output device 150 may be a device such as a display screen, a speaker, etc. that can output data of the electronic device 100 to a user. The input device 160 may be a device such as a keyboard, a mouse, a microphone, or a joystick that can input data to the electronic device 100.

[0059] It should be pointed out that Figure 1 The structure shown in the figure does not constitute a limitation on the computing device, except Figure 1 In addition to the components shown, the computing device may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0060] The training device and application scenario of the target detection network model described in the embodiments of the present application are intended to more clearly illustrate the technical solution of the embodiments of the present application, and do not constitute a limitation on the technical solution provided in the embodiments of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the training device of the target detection network model and the emergence of new business scenarios, the technical solution provided in the embodiments of the present application is also applicable to similar technical problems.

[0061] Next, the training method of the target detection network model is described in detail with reference to the accompanying drawings. Figure 2 A flowchart of a training method for a target detection network model provided in this application. The method is applied to Figure 1 The training device of the target detection network model of the hardware structure shown in the figure specifically includes the following steps:

[0062] Step 210: Acquire a power grid image dataset.

[0063] Step 220 , amplify the power grid image data set to obtain an amplified power grid image data set.

[0064] For a power grid image, two data augmentation methods are used to amplify the power grid image to obtain two amplified images, namely a first sample and a second sample, wherein an intersection-over-union ratio of overlapping areas between the first sample and the second sample is greater than or equal to a threshold.

[0065] For example, the power grid image can be processed by various operations such as flipping, rotating, cropping, deforming, scaling, etc., and the power grid image can also be processed by noise, blurring, color conversion, erasing, filling, etc. to obtain an augmented image. The power grid image in the power grid image dataset is augmented to obtain an augmented power grid image dataset.

[0066] In the embodiment of the present application, the specific method of amplifying the power grid image is not limited, and the above is only an example for explanation.

[0067] Step 230: Determine multi-label information of each power grid image in the augmented power grid image dataset and a first loss corresponding to the multi-label information.

[0068] In the power grid scenario, defective objects and non-defective objects in the power grid image often have similar structures and textures. Defective objects refer to devices that are incomplete in the power grid image. Correspondingly, non-defective objects refer to devices that are complete in the power grid image and whose categories can be clearly determined in the power grid image. However, in target detection in the power grid scenario, defective objects are usually regarded as foreground targets and non-defective images are regarded as background targets. Foreground targets refer to objects of interest in target detection, and background targets refer to objects that users are not interested in during the target detection process, so they are regarded as background. This is significantly different from general datasets such as MS-COCO and Open-Image, in which similar objects are either foreground targets or background targets. Therefore, when training power grid images, deep neural network models that perform well in general datasets regard non-defective objects as background targets. The network tends to focus on the differences between defective objects and non-defective objects and ignore the common features between defective objects and non-defective objects, thereby affecting the model's feature expression ability and generalization ability.

[0069] In addition, there are many more non-defective objects than defective objects in the power grid scene, which will prompt the deep neural network model to ignore the common features between defective objects and non-defective objects, that is, to regard non-defective objects as background targets.

[0070] Therefore, in order to solve the above problems, the present application adds a classification head to the target detection network model for training on a multi-label classification task with low-quality labels. In other words, the multi-label information of each power grid image in the augmented power grid image dataset and the first loss corresponding to the multi-label information are determined.

[0071] First, the defect-free devices in each power grid image in the power grid image data set are marked with a marking frame, wherein the defect-free devices of the same category but different positions in the power grid image use the same marking frame. For example, all the shock-absorbing hammers in a power grid image are marked with a circular frame, and all the equalizing rings are marked with a square frame. It should be noted that the present application does not specifically limit the specific form of the marking frame, and the above-mentioned marking method is only explained as an example.

[0072] Figure 3 A schematic diagram of a power grid image is provided. Figure 3 As shown, the grid image includes anti-vibration hammers, voltage-equalizing rings, insulators, and connecting hardware. The anti-vibration hammer is designed to reduce the vibration of the wire caused by wind. The pole position of the high-voltage overhead line is high and the span is large. When the wire is affected by wind, it will vibrate. When the wire vibrates, the working conditions at the wire suspension point are the most unfavorable. Due to multiple vibrations, the wire will suffer fatigue damage due to periodic bending. When the overhead line span is greater than 120 meters, anti-vibration hammers are generally used for vibration prevention. The working principle of the voltage-equalizing ring is to achieve voltage stability by balancing the voltage in the power system. In the power system, due to different power loads, different power equipment requires different voltages, so the voltage in the power system will be unbalanced. The voltage-equalizing ring achieves voltage stability by balancing the voltage in the power system. An insulator is a device installed between conductors of different potentials or between conductors and grounding components, which can withstand voltage and mechanical stress. In an embodiment of the present application, the defect-free equipment in the grid image is marked by a marking box.

[0073] After marking the defect-free devices in each power grid image in the power grid image data set, the multi-label information of each power grid image in the augmented power grid image data set is determined according to the overlap between each power grid image in the augmented power grid image data set and the marking box of each power grid image in the power grid image data set. In other words, the marked power grid image is compared with each power grid image in the augmented power grid image data set to determine whether each power grid image in the augmented power grid image data set has the device in the marking box in the marked power grid image. For example, the existence of the anti-vibration hammer, grading ring, insulator, connecting hardware, etc. of each power grid image in the augmented power grid image data set is determined. The existence situation can be that the power grid image includes a complete device, or that the power grid image includes half of a device, or that the power grid image does not include a certain device. In the embodiment of the present application, the existence situation is not specifically limited, and the above is only explained as an example. Each device in the power grid image corresponds to a label, and a power grid image can include multiple devices, so a power grid image can have multiple labels.

[0074] Furthermore, the target detection network model of the present application further includes a fully connected layer, which is used to score the power grid image according to the presence of defect-free devices in the power grid image, and determine the first loss corresponding to the multi-label information according to the scoring result of the power grid image. Specifically, according to the formula: L M =XX*Labels+log1+exp-X determines the first loss, where L M represents the first loss, X represents the scoring result of the fully connected layer on the power grid image, and labels represents the multi-label information of the power grid image.

[0075] The first loss of the power grid image obtained in this application is the cross entropy loss. The cross entropy loss is mainly used to measure the difference information between two probability distributions. That is, it is used to measure the difference between the devices in each power grid image in the expanded power grid image dataset and the devices in the annotation box. The cross entropy loss can improve the accuracy of the target detection network model.

[0076] Step 240: construct an object detection network model based on the region proposal network and the R-CNN head for predicting the object category and relative position.

[0077] In order to achieve target detection for power grid images, this application provides a target detection network model. Figure 4 A structural diagram of a target detection network model is provided. Figure 4 It can be seen that the target detection network model includes, in addition to the classification head in step 230, a feature extraction network ResNet, a feature pyramid network FPN, a region proposal network RPN, and an R-CNN head for predicting target categories and relative positions.

[0078] Step 250: determine a total loss function based on the losses of the region proposal network and the R-CNN head and the first loss.

[0079] After the target detection network model is constructed in step 240, the total loss function is determined according to the following formula: L 总 =L rpn_cls L rpn_bbox +L rcnn_cls L rcnn_bbox +r*L M , where L rpn_cls represents the classification loss in the region proposal network, L rpn_bbox represents the bounding box loss in the region proposal network, L rcnn_cls represents the classification loss in the R-CNN head, L rcnn_bbox represents the bounding box loss in the R-CNN head and r represents the weight of the first loss.

[0080] The classification loss is used to determine whether the target detection network model can accurately identify objects in the grid image and classify them into the correct category. The bounding box loss is used to measure the difference between the bounding box predicted by the target detection network model and the true bounding box, which helps ensure that the target detection network model can accurately locate the object.

[0081] Step 260: input the augmented power grid image data set into the target detection network model, and use the total loss function to train the target detection network model until the total loss function converges to obtain the target detection network model.

[0082] As the target detection network model converges, the total loss function of the target detection network model continues to decrease, which means that the defective objects in the power grid image are detected. At this time, the total loss function approaches 0 until it no longer changes, and the target detection network model is obtained.

[0083] The object detection network model trained in this application has the following advantages: First, without directly interfering with the region proposal network RPN and the R-CNN head used to predict the target category and relative position, adding multi-label classification can improve the performance of the backbone network, and the feature alignment of defect images and non-defect images improves the robustness of the model in the face of complex backgrounds. Second, the classification head added to the object detection network model is not coupled with the backbone, which means that the classifier can work well with any other two-stage object detection network or any other backbone network.

[0084] In some embodiments, the present application also provides a target detection method, which is applied to the above target detection device, such as Figure 5 As shown, the method includes:

[0085] Step 510: Acquire an image of the power grid to be detected.

[0086] Among them, the images of the power grid to be inspected are taken by drones during the inspection of transmission lines.

[0087] Step 520: input the power grid image to be detected into the trained target detection network model to obtain the detection result of the power grid image to be detected.

[0088] In the embodiment of the present application, the detection results of the power grid image include 14 categories, such as: insulator self-explosion, porcelain insulator damage, composite insulator damage, discharge gap defect, ground wire clamp tilt, anti-vibration hammer tilt, anti-vibration hammer damage, anti-vibration hammer slippage, equalizing ring damage, equalizing ring tilt, equalizing ring reverse installation, equalizing ring loss, bird's nest, and broken strands.

[0089] It is understandable that in order to implement the functions in the above embodiments, the computer device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0090] Furthermore, as a response to the above Figure 2 The implementation of the method embodiment shown in the figure, the embodiment of the present application provides a training device for a target detection network model. The embodiment of the device corresponds to the aforementioned method embodiment. For ease of reading, this embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspond to all the contents of the aforementioned method embodiment. Specifically, Figure 6 As shown, the training device 600 of the target detection network model includes:

[0091] An acquisition module 610 is used to acquire a power grid image dataset;

[0092] An amplification module 620 is used to amplify the power grid image data set to obtain an amplified power grid image data set;

[0093] A label module 630, configured to determine multi-label information of each power grid image in the augmented power grid image data set and a first loss corresponding to the multi-label information, wherein the multi-label information includes a label of each non-defective device in the power grid image;

[0094] A construction module 640 is used to construct an object detection network model based on a region proposal network and an R-CNN head for predicting object categories and relative positions;

[0095] A determination module 650 for determining a total loss function based on the losses of the region proposal network and the R-CNN head and the first loss;

[0096] The training module 660 is used to input the augmented power grid image data set into the target detection network model, and train the target detection network model using the total loss function until the total loss function converges to obtain the target detection network model.

[0097] Further, such as Figure 6 As shown, the label module 630 is specifically used to label the non-defective devices in each power grid image in the power grid image data set through a labeling frame, wherein the non-defective devices of the same category but different positions in the power grid image use the same labeling frame.

[0098] Further, such as Figure 6As shown, the label module 630 is specifically used to determine the multi-label information of each power grid image in the augmented power grid image dataset according to the overlap between each power grid image in the augmented power grid image dataset and the annotation box of each power grid image in the power grid image dataset.

[0099] Further, such as Figure 6 As shown, the target detection network model also includes a fully connected layer, which is used to score the power grid image according to the presence of non-defective devices in the power grid image. The label module 630 is specifically used to: According to the formula: L M =XX*Labels+log1+exp-X determines the first loss, where L M represents the first loss, X represents the scoring result of the fully connected layer on the power grid image, and labels represents the multi-label information of the power grid image.

[0100] Further, such as Figure 6 As shown, the determination module 650 is specifically used to determine the formula: L 总 =L rpn_cls L rpnb_ box L rcnn_cls +L rc_nbnbox oeLh M Determine the total loss function, where L rpn_cls represents the classification loss in the region proposal network, L rpn_bbox represents the bounding box loss in the region proposal network, L rcnn_cls represents the classification loss in the R-CNN head, L rcnn_bbox represents the bounding box loss in the R-CNN head and r represents the weight of the first loss.

[0101] Furthermore, as a response to the above Figure 5 The implementation of the method embodiment shown in the embodiment of the present application provides a target detection device. The embodiment of the device corresponds to the aforementioned method embodiment. For ease of reading, this embodiment will no longer repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can correspond to all the contents in the aforementioned method embodiment.

[0102] Specific as Figure 7 As shown, the target detection device 700 includes:

[0103] An acquisition module 710 is used to acquire an image of a power grid to be detected;

[0104] The detection module 720 is used to input the power grid image to be detected into the trained target detection network model to obtain the detection result of the power grid image to be detected.

[0105] An embodiment of the present application provides a storage medium on which a program is stored. When the program is executed by a processor, the training method of the target detection network model or the target detection method is implemented.

[0106] An embodiment of the present application provides a processor, which is used to run a program, wherein the program executes the training method of the target detection network model or the target detection method when running.

[0107] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that is initialized with the following method steps: obtaining a power grid image data set; performing an amplification process on the power grid image data set to obtain an amplified power grid image data set; determining multi-label information of each power grid image in the amplified power grid image data set and a first loss corresponding to the multi-label information, wherein the multi-label information includes the labels of each non-defective device in the power grid image; constructing a target detection network model based on a region proposal network and an R-CNN head for predicting target categories and relative positions; determining a total loss function based on the losses of the region proposal network and the R-CNN head and the first loss; inputting the amplified power grid image data set into the target detection network model, and training the target detection network model using the total loss function until the total loss function converges to obtain a target detection network model.

[0108] Furthermore, the defect-free devices in each power grid image in the power grid image dataset are annotated by a labeling box, wherein the defect-free devices of the same category but different positions in the power grid image use the same labeling box.

[0109] Furthermore, according to the overlap between each power grid image in the augmented power grid image data set and the annotation box of each power grid image in the power grid image data set, the multi-label information of each power grid image in the augmented power grid image data set is determined.

[0110] Furthermore, the target detection network model further includes a fully connected layer, which is used to score the power grid image according to the presence of non-defective devices in the power grid image; determining the first loss corresponding to the multi-label information includes: according to the formula: L M =XX*Labels+log1+exp-X determines the first loss, where L M represents the first loss, X represents the scoring result of the fully connected layer on the power grid image, and labels represents the multi-label information of the power grid image.

[0111] Furthermore, according to the formula: L 总 =L rpn_cls +L rpn_bbox +L rcnn_cls +L rcnn_bbox +r*L M Determine the total loss function, where Lrpn_cls represents the classification loss in the region proposal network, L rpn_bbox represents the bounding box loss in the region proposal network, L rcnn_cls represents the classification loss in the R-CNN head, L rcnn_bbox represents the bounding box loss in the R-CNN head and r represents the weight of the first loss.

[0112] The present application also provides another computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes the following method steps: obtaining an image of a power grid to be detected; inputting the image of the power grid to be detected into a trained target detection network model to obtain a detection result of the image of the power grid to be detected.

[0113] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0114] In a typical configuration, the device includes one or more processors (CPU), memory and bus. The device may also include input / output interface, network interface and the like.

[0115] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.

[0116] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0117] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0118] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0119] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A training method for a target detection network model, characterized in that: The method comprises: Obtain a power grid image dataset; Amplifying the power grid image data set to obtain an amplified power grid image data set; Build an object detection network model based on the region proposal network and the R-CNN head for predicting the object category and relative position; Determining multi-label information of each power grid image in the augmented power grid image dataset and a first loss corresponding to the multi-label information, wherein the multi-label information includes a label of each non-defective device in the power grid image, wherein the target detection network model further includes a fully connected layer, and the fully connected layer is used to score the power grid image according to the presence of non-defective devices in the power grid image; Determining a first loss corresponding to the multi-label information includes: According to the formula: L M =XX*Labels+log(1+exp(-X)) determines the first loss, where L M represents the first loss, X represents the scoring result of the fully connected layer on the power grid image, labels represents the multi-label information of the power grid image, wherein labels is the overlap between the annotation boxes of each power grid image in the augmented power grid image dataset and each power grid image in the power grid image dataset; Determining a total loss function based on the losses of the region proposal network and the R-CNN head and the first loss; The augmented power grid image data set is input into the target detection network model, and the target detection network model is trained using the total loss function until the total loss function converges to obtain a final target detection network model.

2. The method according to claim 1, characterized in that The method further comprises: The defect-free devices in each power grid image in the power grid image data set are annotated by a labeling frame, wherein the defect-free devices of the same category but different positions in the power grid image use the same labeling frame.

3. The method according to claim 2, characterized in that Determining multi-label information of each power grid image in the amplified power grid image dataset includes: According to the overlap between each power grid image in the augmented power grid image data set and the annotation box of each power grid image in the power grid image data set, multi-label information of each power grid image in the augmented power grid image data set is determined.

4. The method according to claim 1, characterized in that: Determining a total loss function based on the losses of the region proposal network and the R-CNN head and the first loss includes: According to the formula: L 总 =L rpn_cls +L rpn_bbox +L rcnn_cls +L rcnn_bbox +r*L M Determine the total loss function, where L rpn_cls represents the classification loss in the region proposal network, L rpn_bbox represents the bounding box loss in the region proposal network, L rcnn_cls represents the classification loss in the R-CNN head, L rcnn_bbox represents the bounding box loss in the R-CNN head, and r represents the weight of the first loss.

5. A target detection method, characterized in that: The method comprises: Acquire the image of the power grid to be detected; The power grid image to be detected is input into the final target detection network model obtained by the training method of the target detection network model described in any one of claims 1 to 4 to obtain the detection result of the power grid image to be detected.

6. A training device for a target detection network model, characterized in that: The device comprises: An acquisition module, used for acquiring a power grid image dataset; an amplification module, used for amplifying the power grid image data set to obtain an amplified power grid image data set; A building block for building an object detection network model based on a region proposal network and an R-CNN head for predicting object categories and relative positions; a label module, used to determine multi-label information of each power grid image in the augmented power grid image data set and a first loss corresponding to the multi-label information, wherein the multi-label information includes a label of each non-defective device in the power grid image, wherein the target detection network model further includes a fully connected layer, and the fully connected layer is used to score the power grid image according to the presence of non-defective devices in the power grid image; Determining a first loss corresponding to the multi-label information includes: According to the formula: L M =XX*Labels+log(1+exp(-X)) determines the first loss, where L M represents the first loss, X represents the scoring result of the fully connected layer on the power grid image, labels represents the multi-label information of the power grid image, wherein labels is the overlap between the annotation boxes of each power grid image in the augmented power grid image dataset and each power grid image in the power grid image dataset; A determination module, configured to determine a total loss function based on the losses of the region proposal network and the R-CNN head and the first loss; A training module is used to input the augmented power grid image data set into the target detection network model, and use the total loss function to train the target detection network model until the total loss function converges to obtain a final target detection network model.

7. A target detection device, characterized in that: The device comprises: An acquisition module, used for acquiring an image of a power grid to be detected; A detection module is used to input the power grid image to be detected into the final target detection network model obtained by the training method of the target detection network model described in any one of claims 1-4, so as to obtain the detection result of the power grid image to be detected.

8. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the training method of the target detection network model according to any one of claims 1 to 4 or the target detection method according to claim 5.

9. An electronic device, characterized in that: The device includes at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute the training method of the target detection network model as described in any one of claims 1 to 4 or the target detection method as described in claim 5.

Images