Drop-out fuse operation state detection method and device, medium and terminal

Abstract

The invention discloses a method and a device for detecting the running state of a drop-out fuse, a medium and a terminal, relates to the field of power distribution, and mainly aims to solve the problem of low accuracy of a judgment result of the running state of an existing drop-out fuse. The method includes: acquiring unmanned aerial vehicle inspection image data; identifying a first area where each drop-out fuse is located from the image data based on a fuse identification model, and generating a first operation state detection result of the drop-out fuse; identifying a second area where the fusion tube is located from the image data based on a fusion tube identification model, and mapping the fuse and the fusion tube; and if the first operation state detection result of the drop-out fuse is an abnormal operation state, generating a second operation state detection result of the drop-out fuse according to the ratio of the intersection area between the second region and the first region to the area of the second region, and determining the final operation state detection result of the drop-out fuse as a second operation state detection result.

Description

Technical Field

[0001] The present application relates to the technical field of electric power distribution, and in particular to a method and device, medium, and terminal for detecting the operating status of a drop-out fuse. Background Art

[0002] Drop-out fuses are an important component of the power distribution system. Due to the long-term influence of the natural environment, they are prone to defects such as missing fuse tubes, fuse tube drops, burning and rusting. In order to ensure the safe operation of the distribution line, it is necessary to inspect it regularly to determine its operating status. However, traditional manual inspections or vehicle-mounted inspections are inefficient, costly, and unsafe.

[0003] In recent years, with the development of computer vision technology, more and more power companies have adopted automatic inspection methods to detect the operating status of drop-out fuses. Specifically, they use drones to take images of distribution stations, and then use image recognition technology to automatically detect whether there are defects in the drop-out fuses, thereby determining their operating status.

[0004] However, since the drop-out fuse is located outdoors, the background of the image taken by the drone is complex and has too much redundant information, which can easily lead to the background being mistakenly identified as a drop-out fuse, resulting in low accuracy in the detection results of the operating status of the drop-out fuse; and since the drop-out fuse accounts for a small proportion of the image taken by the drone, the clarity is low, further reducing the accuracy of the detection results. Summary of the invention

[0005] In view of this, the present application provides a method and device, medium, and terminal for detecting the operating status of a drop-type fuse. The main purpose is to improve the existing problem that the accuracy of the judgment results of the operating status of the drop-type fuse is reduced due to the complex background and excessive redundant information of the images taken by the drone, and the small proportion of the drop-type fuse in the images taken by the drone, resulting in low clarity, further reducing the accuracy of the judgment results.

[0006] According to one aspect of the present application, a method for detecting the operating status of a drop-out fuse is provided, comprising:

[0007] Obtain drone inspection image data;

[0008] Based on the fuse recognition model for which model training has been completed, a first region where each drop-out fuse is located is identified from the image data, and a first operating state detection result of each drop-out fuse is generated according to the first region;

[0009] Based on the fuse recognition model for which model training has been completed, a second region where each fuse is located is identified from the image data, and each fuse is mapped to a corresponding fuse according to the position coordinates of the second region in the image data and the position coordinates of the first region in the image data;

[0010] If the first operating state detection result of the drop-out type fuse is an abnormal operating state, the second operating state detection result of the drop-out type fuse is generated based on the ratio of the intersection area between the second area and the corresponding first area to the area of ​​the second area, and the final detection result of the operating state of the drop-out type fuse is determined as the second operating state detection result.

[0011] Preferably, the method further comprises:

[0012] If the first operating state detection result of the drop-out fuse is a normal operating state, the final operating state detection result of the drop-out fuse is determined as a normal operating state.

[0013] Preferably, the method further comprises:

[0014] In the image data, a normal operating state is marked at a first region of a dropout fuse whose first operating state detection result is a normal operating state, and a second operating state detection result is marked at a first region of a dropout fuse whose first operating state detection result is an abnormal operating state.

[0015] Preferably, generating the first operating state detection result of each drop-out fuse according to the first area includes:

[0016] If the fuse tube contained in the first area is in a complete state, it is determined that the first operating state detection result of the drop-out fuse corresponding to the first area is a normal operating state;

[0017] If the fuse tube included in the first area is in an incomplete state, it is determined that the first operating state detection result of the drop-out fuse corresponding to the first area is an abnormal operating state.

[0018] Preferably, generating the second operating state detection result of the drop-out fuse according to the ratio of the intersection area between the second area and the corresponding first area to the area of ​​the second area includes:

[0019] Calculate the ratio of the intersection area between the second region and the corresponding first region to the area of ​​the second region;

[0020] If the proportion is greater than or equal to a preset proportion threshold, it is determined that the second operating state detection result of the drop-out fuse is a fuse tube drop defect;

[0021] If the proportion is less than the preset proportion threshold, it is determined that the second operating state detection result of the drop-out fuse is a fuse tube missing defect.

[0022] Preferably, before the fuse identification model based on the completed model training identifies the first area where each drop-out fuse is located from the image data, the method further includes:

[0023] Acquire multiple UAV inspection historical image data, and mark the standard first area where each drop-out fuse is located in each of the historical image data, and mark the contour area of ​​the fuse tube in each of the standard first areas to obtain a historical image data set;

[0024] Dividing the historical image data set into a training set, a validation set, and a test set according to a preset ratio;

[0025] Using each standard first region contained in each historical image data in the training set to train a backbone feature extraction network to obtain a fuse identification model to be verified;

[0026] Using each standard first area contained in each historical image data in the verification set, the model parameters of the fuse identification model to be verified are adjusted to obtain the fuse identification model to be tested;

[0027] The fuse identification model to be tested is tested using each standard first area contained in each historical image data in the test set. If the test result is qualified, it is determined that the fuse identification model to be tested is a fuse identification model that has completed model training, so as to identify the first area based on the fuse identification model that has completed model training.

[0028] Preferably, before the melt pipe identification model based on the completed model training identifies the second area where each melt pipe is located from the image data, the method further includes:

[0029] The example segmentation model is trained using the contour area of ​​the melt pipe in each standard first area contained in each historical image data in the training set to obtain a melt pipe recognition model to be verified;

[0030] Using the contour area of ​​the molten tube in each standard first area contained in each historical image data in the verification set, the model parameters of the molten tube identification model to be verified are adjusted to obtain the molten tube identification model to be tested;

[0031] The melt tube recognition model to be tested is tested using the contour area of ​​the melt tube in each standard first area contained in each historical image data in the test set. If the test result is qualified, the melt tube recognition model to be tested is determined to be a melt tube recognition model that has completed model training, so as to identify the second area based on the melt tube recognition model that has completed model training.

[0032] According to another aspect of the present application, a device for detecting the operating status of a drop-out fuse is provided, comprising:

[0033] Image data acquisition module, used to acquire UAV inspection image data;

[0034] a first operation status detection result generating module, configured to identify a first region where each drop-out fuse is located from the image data based on a fuse recognition model for which model training has been completed, and to generate a first operation status detection result of each drop-out fuse according to the first region;

[0035] a fuse identification module, configured to identify a second region where each fuse is located from the image data based on the fuse identification model for which model training has been completed, and map each fuse with a corresponding fuse according to the position coordinates of the second region in the image data and the position coordinates of the first region in the image data;

[0036] The final operating status detection result determination module is used to generate the second operating status detection result of the drop-type fuse if the first operating status detection result of the drop-type fuse is an abnormal operating status, based on the proportion of the intersection area between the second area and the corresponding first area to the area of ​​the second area, and determine the final operating status detection result of the drop-type fuse as the second operating status detection result.

[0037] Preferably, the final operation status detection result determination module is further used to:

[0038] If the first operating state detection result of the drop-out fuse is a normal operating state, the final operating state detection result of the drop-out fuse is determined as a normal operating state.

[0039] Preferably, the device further comprises:

[0040] A detection result output module is used to mark the normal operating state in the first area of ​​the drop-out fuse whose first operating state detection result is the normal operating state in the image data, and to mark the second operating state detection result in the first area of ​​the drop-out fuse whose first operating state detection result is the abnormal operating state.

[0041] Preferably, the first operation status detection result generating module is used to:

[0042] If the fuse tube contained in the first area is in a complete state, it is determined that the first operating state detection result of the drop-out fuse corresponding to the first area is a normal operating state;

[0043] If the fuse tube included in the first area is in an incomplete state, it is determined that the first operating state detection result of the drop-out fuse corresponding to the first area is an abnormal operating state.

[0044] Preferably, the final operation status detection result determination module is used to:

[0045] Calculate the ratio of the intersection area between the second region and the corresponding first region to the area of ​​the second region;

[0046] If the proportion is greater than or equal to a preset proportion threshold, it is determined that the second operating state detection result of the drop-out fuse is a fuse tube drop defect;

[0047] If the proportion is less than the preset proportion threshold, it is determined that the second operating state detection result of the drop-out fuse is a fuse tube missing defect.

[0048] Preferably, before the first operation status detection result generating module, the device further includes a fuse identification model training module, which is used to:

[0049] Acquire multiple UAV inspection historical image data, and mark the standard first area where each drop-out fuse is located in each of the historical image data, and mark the contour area of ​​the fuse tube in each of the standard first areas to obtain a historical image data set;

[0050] Dividing the historical image data set into a training set, a validation set, and a test set according to a preset ratio;

[0051] Using each standard first region contained in each historical image data in the training set to train a backbone feature extraction network to obtain a fuse identification model to be verified;

[0052] Using each standard first area contained in each historical image data in the verification set, the model parameters of the fuse identification model to be verified are adjusted to obtain the fuse identification model to be tested;

[0053] The fuse identification model to be tested is tested using each standard first area contained in each historical image data in the test set. If the test result is qualified, it is determined that the fuse identification model to be tested is a fuse identification model that has completed model training, so as to identify the first area based on the fuse identification model that has completed model training.

[0054] Preferably, before the fusion tube identification module, the device further includes a fusion tube identification model training module, which is used to:

[0055] The example segmentation model is trained using the contour area of ​​the melt pipe in each standard first area contained in each historical image data in the training set to obtain a melt pipe recognition model to be verified;

[0056] Using the contour area of ​​the molten tube in each standard first area contained in each historical image data in the verification set, the model parameters of the molten tube identification model to be verified are adjusted to obtain the molten tube identification model to be tested;

[0057] The melt tube recognition model to be tested is tested using the contour area of ​​the melt tube in each standard first area contained in each historical image data in the test set. If the test result is qualified, the melt tube recognition model to be tested is determined to be a melt tube recognition model that has completed model training, so as to identify the second area based on the melt tube recognition model that has completed model training.

[0058] According to another aspect of the present application, a storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to execute operations corresponding to the above-mentioned method for detecting the operating status of a drop-out fuse.

[0059] According to another aspect of the present application, a terminal is provided, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus;

[0060] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned method for detecting the operating status of the drop-out fuse.

[0061] By means of the above technical solution, the technical solution provided by the embodiment of the present application has at least the following advantages:

[0062] The present application provides a method and device, medium, and terminal for detecting the operating status of a drop-out fuse. First, drone inspection image data is acquired; secondly, a first area where each drop-out fuse is located is identified from the image data based on a fuse recognition model that has completed model training, and a first operating status detection result of each drop-out fuse is generated according to the first area respectively; thirdly, a second area where each fuse is located is identified from the image data based on a fuse tube recognition model that has completed model training, and each fuse is mapped to a corresponding fuse tube according to the position coordinates of the second area in the image data and the position coordinates of the first area in the image data; finally, if the first operating status detection result of the drop-out fuse is an abnormal operating status, a second operating status detection result of the drop-out fuse is generated according to the proportion of the intersection area between the second area and the corresponding first area to the area of ​​the second area, and the final detection result of the operating status of the drop-out fuse is determined as the second operating status detection result. Compared with the prior art, the embodiment of the present application first generates a first operating status detection result based on the first area where the drop-out fuse is located for preliminary identification; if it is an abnormal operating state, a second operating status detection result is generated based on the proportion of the intersection area between the second area where the identified fuse is located and the corresponding first area to the area of ​​the second area, and the abnormal operating state is replaced by the second operating status detection result for secondary identification. Since the second area used in the secondary identification takes the fuse as the identification object, the background interference information is ignored, which makes it more focused on the fuse information and avoids the situation where the background is mistakenly identified as a drop-out fuse, thereby improving the accuracy of the detection result of the operating state of the drop-out fuse; at the same time, since the focus is on the fuse information, the situation where the clarity is reduced due to a small proportion is avoided, and the accuracy of the detection result is further improved.

[0063] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0065] Figure 1 A flow chart of a method for detecting the operating status of a drop-out fuse provided in an embodiment of the present application is shown;

[0066] Figure 2 A schematic diagram of a first region provided in an embodiment of the present application is shown;

[0067] Figure 3 A schematic diagram of a second region provided in an embodiment of the present application is shown;

[0068] Figure 4 A flow chart of another method for detecting the operating status of a drop-out fuse provided in an embodiment of the present application is shown;

[0069] Figure 5 A schematic diagram of the detection output result provided by an embodiment of the present application is shown;

[0070] Figure 6 A flowchart of a fuse identification model training provided in an embodiment of the present application is shown;

[0071] Figure 7 A schematic diagram of standard first area marking provided in an embodiment of the present application is shown;

[0072] Figure 8 A schematic diagram of the marking of the contour area of ​​the melting tube provided in an embodiment of the present application is shown;

[0073] Fig. 9 A training flow chart of a melt pipe identification model provided in an embodiment of the present application is shown;

[0074] Fig.10 A block diagram showing a device for detecting the operating status of a drop-out fuse provided in an embodiment of the present application is shown;

[0075] Fig.11 A schematic diagram of the structure of a terminal provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0076] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0077] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0078] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.

[0079] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0080] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0081] The embodiments of the present application can be applied to computer systems / servers, which can operate with many other general or special computing system environments or configurations. Examples of well-known computing systems, environments and / or configurations suitable for use with computer systems / servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0082] Computer systems / servers may be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. In general, program modules may include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers may be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules may be located on local or remote computing system storage media including storage devices.

[0083] The present application embodiment provides a method for detecting the operating status of a drop-out fuse, such as Figure 1 As shown, the method includes:

[0084] 101. Obtain drone inspection image data.

[0085] The drone inspection image data is the image data taken by the drone for the drop-out fuse during the inspection process, which can be a picture taken directly, or a picture taken by first shooting video data and then intercepted from the video data. In the embodiment of the present application, the current execution end can be an operation status monitoring module of the power distribution system.

[0086] 102. Based on the fuse recognition model for which model training has been completed, a first region where each drop-out fuse is located is identified from the image data, and a first operating state detection result of each drop-out fuse is generated according to the first region.

[0087] Among them, the fuse recognition model is obtained by model training based on the backbone feature extraction network, which is used to delineate the area where each drop-out fuse is located from the image data, such as Figure 2 As shown, the area encircled by the box in the figure represents the first area; the first operating state detection result is used to characterize the preliminary detection result for the drop-out fuse, including the normal operating state and the abnormal operating state. In the embodiment of the present application, the first area where each drop-out fuse is located is firstly identified from the image data based on the fuse recognition model, which can be a single drop-out fuse or multiple drop-out fuses. Further, the first operating state detection result of each drop-out fuse is generated according to each first area. It should be noted that when the first operating state detection result is generated, since it is a rough classification, whether the first area contains a complete fuse tube can be used as the classification basis. Specifically, if the first area contains a complete fuse tube, the first operating state detection result of the drop-out fuse is determined to be a normal operating state; if the first area does not contain a complete fuse tube (including a partial fuse tube or no fuse tube), the first operating state detection result of the drop-out fuse is determined to be an abnormal operating state.

[0088] 103. Based on the fuse recognition model that has completed model training, identify the second area where each fuse is located from the image data, and map each fuse with the corresponding fuse according to the position coordinates of the second area in the image data and the position coordinates of the first area in the image data.

[0089] The melt pipe identification model is obtained by model training based on the instance segmentation model, and is used to identify the second area where each melt pipe is located from the image data, such as Figure 3 As shown, the area circled by the box in the figure represents the second area. In the embodiment of the present application, since the fuse recognition model is obtained by model training based on the instance segmentation model, and the instance segmentation model can realize the recognition and positioning of each independent fuse in the image data at the pixel level, it can not only improve the recognition accuracy of the area where the fuse is located, but also distinguish the fuses with overlapping situations. Furthermore, a distance threshold is pre-configured to obtain the position coordinates of the first area where the fuse is located, and the position coordinates of the first area are taken as the origin. If the position coordinates of the second area where the fuse is located exist within the distance threshold, it is considered that the fuse is in a corresponding relationship with the fuse, and at this time, the two are mapped. It should be noted that the position coordinates of the first area and the position coordinates of the second area can be the coordinates of the upper left corner or the lower right corner or the center point of the area, etc., which are not specifically limited in the embodiment of the present application.

[0090] 104. If the first operating state detection result of the drop-type fuse is an abnormal operating state, the second operating state detection result of the drop-type fuse is generated according to the ratio of the intersection area between the second area and the corresponding first area to the area of ​​the second area, and the final detection result of the operating state of the drop-type fuse is determined as the second operating state detection result.

[0091] Among them, the second operating state detection result is used to characterize the detailed detection result for the drop-out type fuse, which may include a fuse drop defect and a fuse missing defect. In the embodiment of the present application, when the first operating state detection result of the drop-out type fuse is an abnormal operating state, it means that the first area does not contain a complete fuse, and may contain a partial fuse or no fuse. At this time, it can be determined whether it contains a partial fuse or no fuse based on the proportion of the intersection area between the second area and the corresponding first area to the area of ​​the second area. Further, when a partial fuse is contained, it is determined that the second operating state detection result of the drop-out type fuse is a fuse drop defect; when there is no fuse, it is determined that the second operating state detection result of the drop-out type fuse is a fuse missing defect. Finally, the fuse drop defect or the fuse missing defect is used to replace the abnormal operating state determined in step 102 of the embodiment.

[0092] It should be noted that compared with directly distinguishing the three operating states of normal operation, fuse drop defect, and fuse missing defect from image data, the embodiment of the present application can effectively improve the accuracy of the drop-type fuse operating state detection results through the two-stage detection in the embodiment steps 102-104.

[0093] Compared with the prior art, the embodiment of the present application first generates a first operating status detection result according to the first area where the drop-type fuse is located for preliminary identification; if it is an abnormal operating state, a second operating status detection result is generated according to the proportion of the intersection area between the second area where the identified fuse is located and the corresponding first area to the area of ​​the second area, and the abnormal operating state is replaced by the second operating status detection result for secondary identification. Since the second area used in the secondary identification takes the fuse as the identification object, the background interference information is ignored, which makes it more focused on the fuse information and avoids the situation where the background is mistakenly identified as a drop-type fuse, thereby improving the accuracy of the detection result of the operating state of the drop-type fuse; at the same time, since it focuses on the fuse information, it avoids the situation where the clarity is reduced due to a small proportion, and further improves the accuracy of the detection result.

[0094] The present application embodiment provides another method for detecting the operating status of a drop-out fuse, such as Figure 4 As shown, the method includes:

[0095] 201. Obtain drone inspection image data.

[0096] The steps of the embodiment of the present application have been described in detail in the aforementioned steps and will not be repeated here.

[0097] 202. Based on the fuse recognition model for which model training has been completed, a first region where each drop-out fuse is located is identified from the image data, and a first operating state detection result of each drop-out fuse is generated according to the first region.

[0098] Specifically, if the fuse contained in the first area is in a complete state, the first operating state detection result of the drop-type fuse corresponding to the first area is determined to be a normal operating state; if the fuse contained in the first area is in an incomplete state, the first operating state detection result of the drop-type fuse corresponding to the first area is determined to be an abnormal operating state.

[0099] The incomplete state includes a state without a melt pipe and a state including a part of the melt pipe.

[0100] 203. Based on the fuse recognition model for which model training has been completed, a second region where each fuse is located is identified from the image data, and each fuse is mapped to a corresponding fuse according to the position coordinates of the second region in the image data and the position coordinates of the first region in the image data.

[0101] The steps of the embodiment of the present application have been described in detail in the aforementioned steps and will not be repeated here.

[0102] 204a. If the first operating state detection result of the drop-type fuse is an abnormal operating state, a second operating state detection result of the drop-type fuse is generated based on the ratio of the intersection area between the second area and the corresponding first area to the area of ​​the second area, and the final detection result of the operating state of the drop-type fuse is determined as the second operating state detection result.

[0103] Specifically, first, the ratio of the intersection area between the second region and the corresponding first region to the area of ​​the second region is calculated, which can be calculated according to the following formula:

[0104]

[0105] Among them, ratio represents the area ratio, Area fuse Indicates the area of ​​the first region, Area tube Indicates the area of ​​the second region. Further, the area ratio calculated above is compared with the preset ratio threshold to determine the specific defect of the abnormal operation state. Specifically, if the ratio is greater than or equal to the preset ratio threshold (such as 0.9), the second operation state detection result of the drop-out fuse is determined to be a fuse drop defect; if the ratio is less than the preset ratio threshold, the second operation state detection result of the drop-out fuse is determined to be a fuse missing defect.

[0106] It should be noted that before calculating the ratio of the intersection area between the second region and the corresponding first region to the area of ​​the second region, the instance segmentation coordinates are first converted into rectangular frame coordinates to calculate the area ratio according to the areas of the two rectangular frames.

[0107] Correspondingly, in step 204a of the embodiment, a second operating state detection result of the drop-out type fuse is generated according to the ratio of the intersection area between the second area and the corresponding first area to the area of ​​the second area, including: calculating the ratio of the intersection area between the second area and the corresponding first area to the area of ​​the second area; if the ratio is greater than or equal to a preset ratio threshold, then determining that the second operating state detection result of the drop-out type fuse is a fuse tube drop defect; if the ratio is less than the preset ratio threshold, then determining that the second operating state detection result of the drop-out type fuse is a fuse tube missing defect.

[0108] 204b. If the first operating state detection result of the drop-out fuse is a normal operating state, the final operating state detection result of the drop-out fuse is determined as a normal operating state.

[0109] In an embodiment of the present application, if the first operating state detection result of the drop-out fuse is a normal operating state, it means that the first area contains a complete fuse tube. At this time, it is considered that the drop-out fuse is in a normal operating state, and the final operating state detection result of the drop-out fuse is determined to be a normal operating state.

[0110] 205. In the image data, a normal operating state is marked at a first region of a drop-out fuse whose first operating state detection result is a normal operating state, and a second operating state detection result is marked at a first region of a drop-out fuse whose first operating state detection result is an abnormal operating state.

[0111] In the embodiment of the present application, after the above two-stage detection, the detection result can be determined as any one of the three types of normal operation, missing fuse defect, and fuse drop defect. At this time, the detection result is marked in the original image data for output, and the output result is as follows: Figure 5 As shown in the figure, the mark in the upper left corner of the box is the detection result.

[0112] In an embodiment of the present application, in order to further define and illustrate, Figure 6 As shown, before step 102 of the embodiment identifies the first area where each drop-out fuse is located from the image data based on the fuse identification model for which model training has been completed, the embodiment method further includes:

[0113] 301. Acquire multiple UAV inspection historical image data, and mark the standard first area where each drop-out fuse is located in each historical image data, and mark the contour area of ​​the fuse tube in each standard first area to obtain a historical image data set.

[0114] 302. Divide the historical image data set into a training set, a validation set, and a test set according to a preset ratio;

[0115] 303. Using each standard first region contained in each historical image data in the training set to train a backbone feature extraction network, a fuse recognition model to be verified is obtained;

[0116] 304. Using each standard first region included in each historical image data in the verification set, adjust the model parameters of the fuse identification model to be verified to obtain the fuse identification model to be tested;

[0117] 305. The fuse recognition model to be tested is tested using each standard first area contained in each historical image data in the test set. If the test result is qualified, the fuse recognition model to be tested is determined to be a fuse recognition model that has completed model training, so as to identify the first area based on the fuse recognition model that has completed model training.

[0118] Steps 301-305 of the embodiment are the training process for the fuse recognition model. Specifically, firstly, a plurality of drone inspection historical image data are obtained, and the image is required to contain a drop-out fuse, and have high definition, little occlusion, and a simple background, and the standard first area where each drop-out fuse is located is marked in each historical image data, such as Figure 7 As shown, and the contour area of ​​the melting tube is marked in each standard first area, such as Figure 8As shown, a historical image data set is obtained. Further, the historical image data set is divided into a training set, a validation set, and a test set according to a preset ratio, for example, 0.7:0.15:0.15, wherein the training set is used to train the fuse recognition model, the validation set is used to verify the fuse recognition model to adjust its model parameters, and the test set is used to test whether the fuse recognition model meets the requirements. Specifically, firstly, the YOLOv7 backbone feature extraction network is trained using each standard first region contained in each historical image data in the training set to obtain a fuse recognition model to be verified, and the training rounds may be 300 rounds; further, each standard first region contained in each historical image data in the validation set is used to adjust the model parameters of the above-mentioned fuse recognition model to be verified to obtain a fuse recognition model to be tested; finally, each standard first region contained in each historical image data in the test set is used to test the above-mentioned fuse recognition model to be tested, and if the test result is qualified, it is determined that the fuse recognition model to be tested is a fuse recognition model that has completed model training.

[0119] In an embodiment of the present application, in order to further define and illustrate, Fig. 9 As shown, before step 103 of the embodiment identifies the second area where each melt tube is located from the image data based on the melt tube recognition model for which model training has been completed, the embodiment method further includes:

[0120] 401. Using the contour area of ​​the molten tube in each standard first area contained in each historical image data in the training set to train the instance segmentation model, a molten tube recognition model to be verified is obtained;

[0121] 402. Adjust the model parameters of the melt pipe identification model to be verified by using the contour area of ​​the melt pipe in each standard first area contained in each historical image data in the verification set to obtain the melt pipe identification model to be tested;

[0122] 403. The melt tube recognition model to be tested is tested using the contour area of ​​the melt tube in each standard first area contained in each historical image data in the test set. If the test result is qualified, the melt tube recognition model to be tested is determined to be the melt tube recognition model for which model training has been completed, so as to recognize the second area based on the melt tube recognition model for which model training has been completed.

[0123] Steps 401-403 of the embodiment are a training process for the melt pipe identification model, specifically, the model training is performed using the contour areas of the melt pipes in each standard first area contained in each historical image data in the training set, the validation set, and the test set in step 302 of the embodiment. Specifically, firstly, the contour areas of the melt pipes in each standard first area contained in each historical image data in the training set are used to train the YOLOv5 instance segmentation model to obtain the melt pipe identification model to be verified, and the training rounds may be 300 rounds; further, the contour areas of the melt pipes in each standard first area contained in each historical image data in the validation set are used to adjust the model parameters of the above-mentioned melt pipe identification model to be verified to obtain the melt pipe identification model to be tested; finally, the contour areas of the melt pipes in each standard first area contained in each historical image data in the test set are used to test the above-mentioned melt pipe identification model to be tested. If the test result is qualified, it is determined that the melt pipe identification model to be tested is the melt pipe identification model that has completed model training.

[0124] The present application provides a method for detecting the operating status of a drop-out fuse. First, unmanned aerial vehicle inspection image data is acquired; secondly, a first area where each drop-out fuse is located is identified from the image data based on a fuse recognition model that has completed model training, and a first operating status detection result of each drop-out fuse is generated according to the first area respectively; thirdly, a second area where each fuse is located is identified from the image data based on a fuse tube recognition model that has completed model training, and each fuse is mapped to a corresponding fuse tube according to the position coordinates of the second area in the image data and the position coordinates of the first area in the image data; finally, if the first operating status detection result of the drop-out fuse is an abnormal operating status, a second operating status detection result of the drop-out fuse is generated according to the proportion of the intersection area between the second area and the corresponding first area to the area of ​​the second area, and the final detection result of the operating status of the drop-out fuse is determined as the second operating status detection result. Compared with the prior art, the embodiment of the present application first generates a first operating status detection result based on the first area where the drop-out fuse is located for preliminary identification; if it is an abnormal operating state, a second operating status detection result is generated based on the proportion of the intersection area between the second area where the identified fuse is located and the corresponding first area to the area of ​​the second area, and the abnormal operating state is replaced by the second operating status detection result for secondary identification. Since the second area used in the secondary identification takes the fuse as the identification object, the background interference information is ignored, which makes it more focused on the fuse information and avoids the situation where the background is mistakenly identified as a drop-out fuse, thereby improving the accuracy of the detection result of the operating state of the drop-out fuse; at the same time, since the focus is on the fuse information, the situation where the clarity is reduced due to a small proportion is avoided, and the accuracy of the detection result is further improved.

[0125] Furthermore, as a response to the above Figure 1 In order to realize the method shown in the figure, the embodiment of the present application provides a device for detecting the operating state of a drop-out fuse, such as Fig.10 As shown, the device comprises:

[0126] An image data acquisition module 51, a first operation status detection result generation module 52, a melting tube identification module 53, and a final operation status detection result determination module 54;

[0127] An image data acquisition module 51 is used to acquire drone inspection image data;

[0128] A first operation status detection result generating module 52, configured to identify a first region where each drop-out fuse is located from the image data based on the fuse recognition model for which model training has been completed, and to generate a first operation status detection result of each drop-out fuse according to the first region;

[0129] a fuse identification module 53, configured to identify the second region where each fuse is located from the image data based on the fuse identification model for which model training has been completed, and map each fuse with the corresponding fuse according to the position coordinates of the second region in the image data and the position coordinates of the first region in the image data;

[0130] The final operating status detection result determination module 54 is used to generate the second operating status detection result of the drop-type fuse if the first operating status detection result of the drop-type fuse is an abnormal operating status, based on the proportion of the intersection area between the second area and the corresponding first area to the area of ​​the second area, and determine the final operating status detection result of the drop-type fuse as the second operating status detection result.

[0131] In a specific application scenario, the final operation status detection result determination module is also used to:

[0132] If the first operating state detection result of the drop-out fuse is a normal operating state, the final operating state detection result of the drop-out fuse is determined as a normal operating state.

[0133] In a specific application scenario, the device further includes:

[0134] A detection result output module is used to mark the normal operating state in the first area of ​​the drop-out fuse whose first operating state detection result is the normal operating state in the image data, and to mark the second operating state detection result in the first area of ​​the drop-out fuse whose first operating state detection result is the abnormal operating state.

[0135] In a specific application scenario, the first operation status detection result generating module is used to:

[0136] If the fuse tube contained in the first area is in a complete state, it is determined that the first operating state detection result of the drop-out fuse corresponding to the first area is a normal operating state;

[0137] If the fuse tube included in the first area is in an incomplete state, it is determined that the first operating state detection result of the drop-out fuse corresponding to the first area is an abnormal operating state.

[0138] In a specific application scenario, the final operation status detection result determination module is used to:

[0139] Calculate the ratio of the intersection area between the second region and the corresponding first region to the area of ​​the second region;

[0140] If the proportion is greater than or equal to a preset proportion threshold, it is determined that the second operating state detection result of the drop-out fuse is a fuse tube drop defect;

[0141] If the proportion is less than the preset proportion threshold, it is determined that the second operating state detection result of the drop-out fuse is a fuse tube missing defect.

[0142] In a specific application scenario, before the first operation status detection result generating module, the device further includes a fuse identification model training module, which is used to:

[0143] Acquire multiple UAV inspection historical image data, and mark the standard first area where each drop-out fuse is located in each of the historical image data, and mark the contour area of ​​the fuse tube in each of the standard first areas to obtain a historical image data set;

[0144] Dividing the historical image data set into a training set, a validation set, and a test set according to a preset ratio;

[0145] Using each standard first region contained in each historical image data in the training set to train a backbone feature extraction network to obtain a fuse identification model to be verified;

[0146] Using each standard first area contained in each historical image data in the verification set, the model parameters of the fuse identification model to be verified are adjusted to obtain the fuse identification model to be tested;

[0147] The fuse identification model to be tested is tested using each standard first area contained in each historical image data in the test set. If the test result is qualified, it is determined that the fuse identification model to be tested is a fuse identification model that has completed model training, so as to identify the first area based on the fuse identification model that has completed model training.

[0148] In a specific application scenario, before the molten tube identification module, the device further includes a molten tube identification model training module, which is used to:

[0149] The example segmentation model is trained using the contour area of ​​the melt pipe in each standard first area contained in each historical image data in the training set to obtain a melt pipe recognition model to be verified;

[0150] Using the contour area of ​​the molten tube in each standard first area contained in each historical image data in the verification set, the model parameters of the molten tube identification model to be verified are adjusted to obtain the molten tube identification model to be tested;

[0151] The melt tube recognition model to be tested is tested using the contour area of ​​the melt tube in each standard first area contained in each historical image data in the test set. If the test result is qualified, the melt tube recognition model to be tested is determined to be a melt tube recognition model that has completed model training, so as to identify the second area based on the melt tube recognition model that has completed model training.

[0152] The present application provides a detection device for the operating status of a drop-out fuse. First, unmanned aerial vehicle inspection image data is acquired; secondly, a first area where each drop-out fuse is located is identified from the image data based on a fuse recognition model that has completed model training, and a first operating status detection result of each drop-out fuse is generated according to the first area respectively; thirdly, a second area where each fuse is located is identified from the image data based on a fuse tube recognition model that has completed model training, and each fuse is mapped to a corresponding fuse tube according to the position coordinates of the second area in the image data and the position coordinates of the first area in the image data; finally, if the first operating status detection result of the drop-out fuse is an abnormal operating status, a second operating status detection result of the drop-out fuse is generated according to the proportion of the intersection area between the second area and the corresponding first area to the area of ​​the second area, and the final detection result of the operating status of the drop-out fuse is determined as the second operating status detection result. Compared with the prior art, the embodiment of the present application first generates a first operating status detection result based on the first area where the drop-out fuse is located for preliminary identification; if it is an abnormal operating state, a second operating status detection result is generated based on the proportion of the intersection area between the second area where the identified fuse is located and the corresponding first area to the area of ​​the second area, and the abnormal operating state is replaced by the second operating status detection result for secondary identification. Since the second area used in the secondary identification takes the fuse as the identification object, the background interference information is ignored, which makes it more focused on the fuse information and avoids the situation where the background is mistakenly identified as a drop-out fuse, thereby improving the accuracy of the detection result of the operating state of the drop-out fuse; at the same time, since the focus is on the fuse information, the situation where the clarity is reduced due to a small proportion is avoided, and the accuracy of the detection result is further improved.

[0153] According to an embodiment of the present application, a storage medium is provided, wherein the storage medium stores at least one executable instruction, and the computer executable instruction can execute the method for detecting the operating status of a drop-out fuse in any of the above method embodiments.

[0154] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0155] Fig.11 A schematic diagram of the structure of a terminal provided according to an embodiment of the present application is shown, and the specific embodiment of the present application does not limit the specific implementation of the terminal.

[0156] like Fig.11 As shown, the terminal may include: a processor (processor) 602 , a communication interface (Communications Interface) 604 , a memory (memory) 606 , and a communication bus 608 .

[0157] The processor 602 , the communication interface 604 , and the memory 606 communicate with each other via a communication bus 608 .

[0158] The communication interface 604 is used to communicate with other devices such as clients or other servers.

[0159] The processor 602 is used to execute the program 610, and specifically can execute the relevant steps in the above-mentioned embodiment of the method for detecting the operating status of the drop-out fuse.

[0160] Specifically, the program 610 may include program codes, which include computer operation instructions.

[0161] The processor 602 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the computer device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0162] The memory 606 is used to store the program 610. The memory 606 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0163] The program 610 may be specifically configured to enable the processor 602 to perform the following operations:

[0164] Obtain drone inspection image data;

[0165] Based on the fuse recognition model for which model training has been completed, a first region where each drop-out fuse is located is identified from the image data, and a first operating state detection result of each drop-out fuse is generated according to the first region;

[0166] Based on the fuse tube recognition model for which model training has been completed, a second region where each tube is located is identified from the image data, and each fuse is mapped to a corresponding fuse tube according to the position coordinates of the second region in the image data and the position coordinates of the first region in the image data;

[0167] If the first operating state detection result of the drop-out type fuse is an abnormal operating state, the second operating state detection result of the drop-out type fuse is generated based on the ratio of the intersection area between the second area and the corresponding first area to the area of ​​the second area, and the final detection result of the operating state of the drop-out type fuse is determined as the second operating state detection result.

[0168] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device of the above-mentioned method for detecting the operating status of the drop-out fuse, and supports the operation of the information processing program and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.

[0169] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0170] The method and system of the present application may be implemented in many ways. For example, the method and system of the present application may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present application are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present application may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers a recording medium storing a program for executing the method according to the present application.

[0171] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0172] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting the operating status of a drop-out fuse, characterized in that: include: Obtain drone inspection image data; Based on the fuse recognition model for which model training has been completed, a first region where each drop-out fuse is located is identified from the image data, and a first operating state detection result of each drop-out fuse is generated according to the first region; Based on the fuse recognition model for which model training has been completed, a second region where each fuse is located is identified from the image data, and each fuse is mapped to a corresponding fuse according to the position coordinates of the second region in the image data and the position coordinates of the first region in the image data; If the first operating state detection result of the drop-out type fuse is an abnormal operating state, the second operating state detection result of the drop-out type fuse is generated based on the ratio of the intersection area between the second area and the corresponding first area to the area of ​​the second area, and the final detection result of the operating state of the drop-out type fuse is determined as the second operating state detection result.

2. The method according to claim 1, characterized in that The method further comprises: If the first operating state detection result of the drop-out fuse is a normal operating state, the final operating state detection result of the drop-out fuse is determined as a normal operating state.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: In the image data, a normal operating state is marked at a first region of a dropout fuse whose first operating state detection result is a normal operating state, and a second operating state detection result is marked at a first region of a dropout fuse whose first operating state detection result is an abnormal operating state.

4. The method according to claim 1, characterized in that The generating the first operating state detection result of each of the drop-out fuses according to the first region includes: If the fuse tube contained in the first area is in a complete state, it is determined that the first operating state detection result of the drop-out fuse corresponding to the first area is a normal operating state; If the fuse tube included in the first area is in an incomplete state, it is determined that the first operating state detection result of the drop-out fuse corresponding to the first area is an abnormal operating state.

5. The method according to claim 1, characterized in that The generating a second operating state detection result of the drop-out fuse according to a ratio of an intersection area between the second area and the corresponding first area to an area of ​​the second area includes: Calculate the ratio of the intersection area between the second region and the corresponding first region to the area of ​​the second region; If the proportion is greater than or equal to a preset proportion threshold, it is determined that the second operating state detection result of the drop-out fuse is a fuse tube drop defect; If the proportion is less than the preset proportion threshold, it is determined that the second operating state detection result of the drop-out fuse is a fuse tube missing defect.

6. The method according to claim 1, characterized in that Before the fuse identification model based on the completed model training identifies the first area where each drop-out fuse is located from the image data, the method further includes: Acquire multiple UAV inspection historical image data, and mark the standard first area where each drop-out fuse is located in each of the historical image data, and mark the contour area of ​​the fuse tube in each of the standard first areas to obtain a historical image data set; Dividing the historical image data set into a training set, a validation set, and a test set according to a preset ratio; Using each standard first region contained in each historical image data in the training set to train a backbone feature extraction network to obtain a fuse identification model to be verified; Using each standard first area contained in each historical image data in the verification set, the model parameters of the fuse identification model to be verified are adjusted to obtain the fuse identification model to be tested; The fuse identification model to be tested is tested using each standard first area contained in each historical image data in the test set. If the test result is qualified, it is determined that the fuse identification model to be tested is a fuse identification model that has completed model training, so as to identify the first area based on the fuse identification model that has completed model training.

7. The method according to claim 6, characterized in that Before the melt pipe identification model based on the completed model training identifies the second area where each melt pipe is located from the image data, the method further includes: The example segmentation model is trained using the contour area of ​​the melt pipe in each standard first area contained in each historical image data in the training set to obtain a melt pipe recognition model to be verified; Using the contour area of ​​the molten tube in each standard first area contained in each historical image data in the verification set, the model parameters of the molten tube identification model to be verified are adjusted to obtain the molten tube identification model to be tested; The melt tube recognition model to be tested is tested using the contour area of ​​the melt tube in each standard first area contained in each historical image data in the test set. If the test result is qualified, the melt tube recognition model to be tested is determined to be a melt tube recognition model that has completed model training, so as to identify the second area based on the melt tube recognition model that has completed model training.

8. A device for detecting the operating status of a drop-out fuse, characterized in that: include: Image data acquisition module, used to acquire UAV inspection image data; a first operation status detection result generating module, configured to identify a first region where each drop-out fuse is located from the image data based on a fuse recognition model for which model training has been completed, and to generate a first operation status detection result of each drop-out fuse according to the first region; a fuse identification module, configured to identify a second region where each fuse is located from the image data based on the fuse identification model for which model training has been completed, and map each fuse with a corresponding fuse according to the position coordinates of the second region in the image data and the position coordinates of the first region in the image data; The final operating status detection result determination module is used to generate the second operating status detection result of the drop-type fuse if the first operating status detection result of the drop-type fuse is an abnormal operating status, based on the proportion of the intersection area between the second area and the corresponding first area to the area of ​​the second area, and determine the final operating status detection result of the drop-type fuse as the second operating status detection result.

9. A storage medium, wherein at least one executable instruction is stored in the storage medium, characterized in that: The executable instructions enable the processor to execute operations corresponding to the method for detecting the operating status of a drop-out fuse according to any one of claims 1 to 7.

10. A terminal, comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, wherein the executable instruction enables the processor to execute an operation corresponding to the method for detecting the operating status of a drop-out fuse according to any one of claims 1-7.

Images