Foreign matter monitoring method, foreign matter monitoring device, electronic equipment and storage medium

Abstract

The invention discloses a foreign matter monitoring method, a foreign matter monitoring device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a scene image of a current transformer substation; the current substation scene image is input to a target foreign matter monitoring model, a corresponding foreign matter monitoring result is obtained, and the target foreign matter monitoring model is obtained based on a Faster R-CNN model, a non-maximum suppression module and a Gaussian weighting function. According to the method, foreign matter monitoring is carried out on the transformer substation based on the target foreign matter monitoring model constructed based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weighting function, so that misjudgment on the foreign matter of the transformer substation due to the influence of the fatigue degree of inspectors on manual inspection is avoided; and the monitoring accuracy and the monitoring efficiency of foreign matter monitoring on the transformer substation are improved.

Description

Technical Field

[0001] The present application belongs to the technical field of substation monitoring, and in particular, relates to a foreign matter monitoring method, a foreign matter monitoring device, an electronic device and a storage medium. Background Art

[0002] Substations are places in power systems that transform voltage and current, receive and distribute electric energy, and are a vital component of power systems. During the operation of substations, when foreign objects appear in the substations, such as animals that accidentally enter, dropped hanging objects (plastic films, kites, etc.), and related equipment left by staff, it is easy to cause substation line failures and increase the safety risks of substation operation. Therefore, it is urgent to monitor foreign objects in substations.

[0003] At present, foreign object monitoring in substations mainly relies on manual inspections, which are affected by the fatigue of inspectors and are prone to misjudgment, resulting in low accuracy in foreign object monitoring of substations. Summary of the invention

[0004] In view of this, embodiments of the present application provide a foreign matter monitoring method, a foreign matter monitoring device, an electronic device and a storage medium to overcome the above problems of the prior art.

[0005] In the first aspect, an embodiment of the present application provides a foreign object monitoring method, including: obtaining a current substation scene image; inputting the current substation scene image into a target foreign object monitoring model to obtain a corresponding foreign object monitoring result, wherein the target foreign object monitoring model is obtained based on a Faster R-CNN model, a non-maximum suppression module and a Gaussian weight function.

[0006] Among them, in some optional embodiments, before obtaining the current substation scene image, the foreign object monitoring method also includes: constructing an initial foreign object monitoring model based on the Faster R-CNN model, non-maximum suppression module and Gaussian weight function; training the initial foreign object monitoring model to obtain the target foreign object monitoring model.

[0007] Among them, in some optional embodiments, the initial foreign body monitoring model is constructed based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function, including: constructing a target suppression module based on the non-maximum suppression module and the Gaussian weight function; fusing the region of interest pooling layer of the Faster R-CNN model and the target suppression module to obtain the initial foreign body monitoring model.

[0008] Among them, in some optional embodiments, the training of the initial foreign object monitoring model to obtain the target foreign object monitoring model includes: obtaining a historical substation scene image; obtaining a corresponding sample set based on the historical substation scene image, and the sample set at least includes an initial training set; inputting the initial training set into the initial foreign object monitoring model for training to obtain the target foreign object monitoring model.

[0009] Among them, in some optional embodiments, before the inputting of the initial training set to the initial foreign body monitoring model for training to obtain the target foreign body monitoring model, the foreign body monitoring method also includes: performing data enhancement processing on the initial training set to obtain an enhanced training set; the inputting of the initial training set to the initial foreign body monitoring model for training to obtain the target foreign body monitoring model includes: inputting the enhanced training set to the initial foreign body monitoring model for training to obtain the target foreign body monitoring model.

[0010] Among them, in some optional embodiments, the sample set also includes a test set, and before obtaining the current substation scene image, the foreign object monitoring method also includes: determining whether the target foreign object monitoring model converges according to the test set; obtaining the current substation scene image includes: when it is determined that the target foreign object monitoring model converges according to the test set, obtaining the current substation scene image.

[0011] Among them, in some optional embodiments, before inputting the current substation scene image into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result, the foreign object monitoring method also includes: performing sliding slicing processing on the current substation scene image to obtain multiple slice images; inputting the current substation scene image into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result includes: inputting the multiple slice images and the current substation scene image into the target foreign object monitoring model to obtain the foreign object monitoring result.

[0012] In a second aspect, an embodiment of the present application provides a foreign body monitoring device, which includes an acquisition module and an input module. The acquisition module is used to acquire a current substation scene image; the input module is used to input the current substation scene image into a target foreign body monitoring model to obtain a corresponding foreign body monitoring result, and the target foreign body monitoring model is obtained based on a Faster R-CNN model, a non-maximum suppression module and a Gaussian weight function.

[0013] In a third aspect, an embodiment of the present application provides an electronic device comprising a memory; one or more processors coupled to the memory; and one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the foreign object monitoring method provided in the first aspect above.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored. The program code can be called by a processor to execute the foreign matter monitoring method provided in the first aspect above.

[0015] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device executes the foreign matter monitoring method provided in the first aspect above.

[0016] The solution provided by the present application obtains the current substation scene image and inputs the current substation scene image into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result. The target foreign object monitoring model is obtained based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function. The target foreign object monitoring model constructed based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function is used to monitor the substation for foreign objects, thereby avoiding the misjudgment of foreign objects in the substation caused by the fatigue of the inspectors during manual patrol inspections, and improving the monitoring accuracy and efficiency of foreign object monitoring in the substation.

[0017] Furthermore, the non-maximum suppression module is improved based on the Gaussian weight function, so that smaller foreign objects can be easily identified by the target foreign object monitoring model, avoiding the missed detection of smaller foreign objects by the target foreign object monitoring model, which is conducive to further improving the monitoring accuracy of foreign objects in substations. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 A schematic diagram of a scenario of a foreign object monitoring system provided in an embodiment of the present application is shown.

[0020] Figure 2A schematic flow chart of a foreign body monitoring method provided in an embodiment of the present application is shown.

[0021] Figure 3 Another flow chart of the foreign matter monitoring method provided in an embodiment of the present application is shown.

[0022] Figure 4 A structural schematic diagram of a target foreign matter monitoring model in the foreign matter monitoring method provided in an embodiment of the present application is shown.

[0023] Figure 5 Another flow chart of the foreign matter monitoring method provided in the embodiment of the present application is shown.

[0024] Figure 6 A structural block diagram of a foreign matter monitoring device provided in an embodiment of the present application is shown.

[0025] Figure 7 A functional block diagram of an electronic device provided in an embodiment of the present application is shown.

[0026] Figure 8 A computer-readable storage medium provided in an embodiment of the present application for storing or carrying a program code for implementing a foreign matter monitoring method provided in an embodiment of the present application is shown.

[0027] Fig. 9 A computer program product provided in an embodiment of the present application and used for storing or carrying program codes for implementing the foreign matter monitoring method provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0030] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0031] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0032] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0033] Substations are places in power systems that transform voltage and current, receive and distribute electric energy, and are a vital component of power systems. During the operation of substations, when foreign objects appear in the substations, such as animals that accidentally enter, dropped hanging objects (plastic films, kites, etc.), and related equipment left by staff, it is easy to cause substation line failures and increase the safety risks of substation operation. Therefore, it is urgent to monitor foreign objects in substations.

[0034] At present, foreign body monitoring in substations mainly relies on manual inspections, which are affected by the fatigue of inspectors and are prone to misjudgment, resulting in low accuracy of foreign body monitoring in substations. In addition, the manual inspection method leads to low efficiency of foreign body monitoring in substations.

[0035] In response to the above problems, the foreign object monitoring method, foreign object monitoring device, electronic device and storage medium provided in the embodiments of the present application obtain the current substation scene image and input the current substation scene image into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result. The target foreign object monitoring model is obtained based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function, and the target foreign object monitoring model constructed based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function is used to monitor the substation for foreign objects, thereby avoiding the misjudgment of foreign objects in the substation caused by the fatigue of the inspectors during manual patrol inspections, and improving the monitoring accuracy and efficiency of foreign object monitoring in the substation.

[0036] Furthermore, the non-maximum suppression module is improved based on the Gaussian weight function, so that smaller foreign objects can be easily identified by the target foreign object monitoring model, avoiding the missed detection of smaller foreign objects by the target foreign object monitoring model, which is conducive to further improving the monitoring accuracy of foreign objects in substations.

[0037] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0038] See also Figure 1 , which shows a schematic diagram of an application scenario of the foreign object monitoring system provided in an embodiment of the present application, which may include a substation 100, a camera 200 and a processing device 300. The camera 200 is installed at a position where it can collect substation scene images of the substation 100. The camera 200 is connected to the processing device 300 via a network and exchanges data with the processing device 300 via the network.

[0039] Among them, the substation 100 may include a variety of substation equipment. For example, the various substation equipment may include transformers, circuit breakers, capacitors, high-voltage switchgear, mutual inductors, compensation device protection devices and relay protection devices, etc. The type of substation equipment is not limited here and can be set according to actual needs.

[0040] The camera 200 may be any one of a wide-angle camera, a macro camera, an ultra-wide-angle camera or a panoramic camera, etc. The type of the camera 200 is not limited here and may be specifically set according to actual needs.

[0041] The processing device 300 may be a terminal device or a server, etc. The type of the processing device 300 is not limited here and may be specifically configured according to actual needs.

[0042] The terminal device can be a mobile terminal device (for example, a mobile phone, a personal digital assistant (PDA), a tablet computer (Tablet Personal Computer, Tablet PC), a laptop computer, a smart watch, a smart bracelet or a wearable device, etc.), or a fixed terminal device (desktop computer, smart panel, etc.), etc.

[0043] The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or any one of the cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), big data or artificial intelligence platforms, etc.

[0044] The network may be any one of a ZigBee network, a Bluetooth (BT) network, a Wireless Fidelity (Wi-Fi) network, a Thread network, a Long Range Radio (LoRa) network, a Low-Power Wide-Area Network (LPWAN), an infrared network, a Narrow Band Internet of Things (NB-IoT), a Controller Area Network (CAN), a Digital Living Network Alliance (DLNA) network, a Wide Area Network (WAN), a Local Area Network (LAN), a Metropolitan Area Network (MAN), or a Wireless Personal Area Network (WPAN), etc., without limitation here.

[0045] See also Figure 2 , which shows a flow chart of a foreign body monitoring method provided by an embodiment of the present application. In a specific embodiment, the foreign body monitoring method can be applied to Figure 1 The processing device 300 in the foreign body monitoring system shown in FIG. Figure 2 The process shown is described in detail, and the foreign matter monitoring method may include the following steps 110 and 120.

[0046] Step 110: Acquire the current substation scene image.

[0047] In an embodiment of the present application, when inspection personnel need to monitor the substation for foreign objects, they can send monitoring instructions to the processing device, the processing device receives and responds to the monitoring instructions, sends image acquisition instructions to the camera through the network, the camera receives and responds to the image acquisition instructions, performs scene image acquisition on the substation, obtains the current substation scene image, and sends the current substation scene image to the processing device through the network, and the processing device receives the current substation scene image returned by the camera.

[0048] In some embodiments, the processing device can detect the operation of the inspector, and when it is determined that the inspector has input a monitoring instruction for monitoring the substation for foreign objects based on the detected operation of the inspector, the monitoring instruction for monitoring the substation for foreign objects is received. For example, when the inspector needs to monitor the substation for foreign objects, he can perform a touch operation on the operation panel of the processing device, and the processing device responds to the touch operation of the inspector, generates a corresponding touch signal, and analyzes the touch signal. When it is determined that the touch signal is a preset monitoring signal for representing the monitoring of foreign objects in the substation, it is determined that the monitoring instruction for monitoring the substation for foreign objects is received.

[0049] In some embodiments, the processing device may be provided with a voice recognition module. When the inspectors need to monitor the substation for foreign objects, the inspectors may send voice information within the voice collection range of the voice recognition module. The voice recognition module collects the voice information sent by the inspectors, and performs voice recognition on the collected voice information. Based on the recognition results of the voice recognition, it is determined that the recognition results contain keywords for indicating foreign object monitoring of the substation, such as "substation foreign object monitoring", and for example, "substation" and "foreign object monitoring", etc., and it is determined that a monitoring instruction for foreign object monitoring of the substation has been received.

[0050] As an example, the voice message sent by the inspection personnel is: monitor the substation for foreign objects, and the recognition result of the voice recognition contains the keywords "substation" and "foreign object monitoring", then it is determined that the monitoring instruction for monitoring the substation for foreign objects has been received.

[0051] In some embodiments, the foreign matter monitoring system may further include a client, which is connected to the processing device via a network and exchanges data with the processing device via the network.

[0052] When the inspection personnel need to monitor the substation for foreign objects, they can send monitoring instructions to the client. The client receives and responds to the monitoring instructions and forwards the monitoring instructions to the processing device through the network. The processing device receives the monitoring instructions forwarded by the client.

[0053] Among them, the client can be any one of a mobile client (for example, a mobile phone client, a PDA client, a Tablet PC client, a laptop client, a smart watch client, a smart bracelet client or a wearable client, etc.) or a fixed client (for example, a desktop computer client, a smart panel client, etc.). The type of client is not limited here and can be set according to actual needs.

[0054] Step 120: Input the current substation scene image into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result.

[0055] In an embodiment of the present application, after the processing device obtains the current substation scene image, the current substation scene image can be input into the target foreign object monitoring model. The target foreign object monitoring model receives and responds to the current substation scene image, performs foreign object monitoring on the substation according to the current substation scene image, and obtains the corresponding foreign object monitoring result. The target foreign object monitoring model is obtained based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function, and realizes the foreign object monitoring of the substation by the target foreign object monitoring model constructed based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function, thereby avoiding the misjudgment of foreign objects in the substation due to the fatigue of the inspectors during manual patrol inspections, and improving the monitoring accuracy and efficiency of foreign object monitoring of the substation.

[0056] Furthermore, the non-maximum suppression module is improved based on the Gaussian weight function, so that smaller foreign objects can be easily identified by the target foreign object monitoring model, avoiding the missed detection of smaller foreign objects by the target foreign object monitoring model, which is conducive to further improving the monitoring accuracy of foreign objects in substations.

[0057] The foreign object monitoring result may include a first monitoring result used to characterize that the current substation scene image does not contain foreign object information, and a second monitoring result used to characterize that the current substation scene image contains foreign object information.

[0058] Foreign object information may include the type of foreign object (for example, animals, hanging objects, related equipment, etc.), the location of the foreign object, the size of the foreign object, etc. The type of foreign object information is not limited here and can be set according to actual needs.

[0059] In some embodiments, after the processing device obtains the current substation scene image, it can perform sliding slicing processing on the current substation scene image to obtain multiple slice images, and input the multiple slice images and the current substation scene image to the target foreign body monitoring model. The target foreign body monitoring model receives and responds to the multiple slice images and the current substation scene image, performs foreign body monitoring on the substation according to the multiple slice images and the current substation scene image, obtains the foreign body monitoring result, and performs foreign body monitoring on the substation according to the multiple slice images obtained by sliding slicing the current substation scene image, which can avoid the loss of small target foreign body information caused by cropping the current substation scene image in the process of performing foreign body monitoring on the substation according to the current substation scene image, thereby improving the detection rate of small target foreign bodies in the substation. And by inputting the multiple slice images and the current substation scene image into the target foreign body monitoring model together, the detected small target foreign bodies can be mapped back to the current substation scene image, which is convenient for the inspection personnel to quickly locate the position of the small target foreign bodies.

[0060] Among them, any two adjacent sliced images among multiple sliced images partially overlap. The sliding slicing retains the partial information repetition of any two adjacent sliced images, which can prevent small target foreign objects from being cut at the edges and thus not being detected, and is beneficial to improving the detection rate of small target foreign objects in the substation.

[0061] In an application scenario, the size of the current substation scene image is H pixels * W pixels, and the slicing window for sliding slicing processing is h pixels * w pixels. That is, the current substation scene image is subjected to sliding slicing processing with a slicing window of h pixels * w pixels to obtain multiple sliced images.

[0062] The sliding step of the slicing window is set to s, where s < h and s < w, which can ensure that any two adjacent sliced images among multiple sliced images partially overlap.

[0063] In the W-pixel direction of the current substation scene image, the number of sliced images obtained by sliding slicing processing is m, where is the ceiling symbol.

[0064] In the H-pixel direction of the current substation scene image, the number of sliced images obtained by sliding slicing processing is n,

[0065] Finally, the number of multiple sliced images after the current substation scene image is subjected to sliding slicing processing is m * n.

[0066] In some embodiments, the foreign object monitoring result can be the second monitoring result. After the processing device inputs the current substation scene image into the target foreign object monitoring model and obtains the corresponding second monitoring result, a warning message can be generated so that the inspection personnel can conduct inspections and handling of the foreign objects in the substation according to the warning message, which is beneficial to improving the operation safety of the power system.

[0067] Among them, the warning message can be at least any one of a sound warning message, a text warning message, a light warning message, etc. The type of the warning message is limited here and can be specifically set according to actual needs.

[0068] The solution provided by this application, by acquiring the current substation scene image and inputting the current substation scene image into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result, where the target foreign object monitoring model is obtained based on the Faster R-CNN model, the non-maximum suppression module, and the Gaussian weight function, realizes the foreign object monitoring of the substation by the target foreign object monitoring model constructed based on the Faster R-CNN model, the non-maximum suppression module, and the Gaussian weight function, avoids the misjudgment of foreign objects in the substation caused by the influence of the fatigue degree of the inspection personnel in manual patrol inspections, and improves the monitoring accuracy and monitoring efficiency of foreign object monitoring in the substation.

[0069] Furthermore, the non-maximum suppression module is improved based on the Gaussian weight function, so that smaller foreign objects can be easily identified by the target foreign object monitoring model, avoiding the missed detection of smaller foreign objects by the target foreign object monitoring model, which is conducive to further improving the monitoring accuracy of foreign objects in substations.

[0070] See also Figure 3 , which shows a flow chart of a foreign body monitoring method provided by another embodiment of the present application. In a specific embodiment, the foreign body monitoring method can be applied to Figure 1 The processing device 300 in the foreign body monitoring system shown in FIG. Figure 3 The process shown is described in detail, and the foreign matter monitoring method may include the following steps 210 and 240.

[0071] Step 210: construct an initial foreign body monitoring model based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function.

[0072] In this embodiment, the processing device can construct a target suppression module based on a non-maximum suppression (NMS) module and a Gaussian weight function, and fuse the region of interest (ROI) pooling layer and the target suppression module of the Faster R-CNN model to obtain an initial foreign object monitoring model. The NMS module is improved based on the Gaussian weight function, so that smaller foreign objects can be easily identified by the target foreign object monitoring model, thereby avoiding the omission of smaller foreign objects by the target foreign object monitoring model, which is conducive to further improving the monitoring accuracy of foreign objects in substations.

[0073] The initial foreign body monitoring model may include a backbone network, a feature mapping layer, a region proposal network (RPN), a ROI pooling layer, a target suppression module, and a classification module, such as Figure 4 shown.

[0074] The feature mapping layer is connected between the backbone network and the ROI pooling layer, the RPN network is connected between the feature mapping layer and the ROI pooling layer, and the target suppression module is connected between the ROI pooling layer and the classification module.

[0075] The backbone network is used to extract features from the input image to generate a feature map. The backbone network can be VGG16. The backbone network can be composed of 13 convolutional layers, 5 pooling layers and 3 fully connected layers. Each convolutional layer can be connected to a rectified linear unit layer.

[0076] The RPN network can be used to predict the target area at each position in the input image. During the training process, the RPN network slides along the convolutional feature map generated by the last shared convolutional layer with a 3×3 spatial window.

[0077] The ROI pooling layer can be used to output proposal boxes of different sizes into images of fixed size. The ROI pooling layer can also be used to collect feature maps and extract proposal feature maps for subsequent classification and regression.

[0078] The target suppression module uses a Gaussian weight function to optimize the NMS module. The NMS module uses the initial NMS algorithm to eliminate redundant bounding boxes. The initial NMS algorithm is shown in Formula 1.

[0079] Formula 1 is:

[0080] Among them, s i is the confidence of the detection box corresponding to the category, M is the detection box with the highest confidence, IOU(M,b i ) is the intersection-and-union ratio, N t is the first IOU threshold, b i Represents the target box.

[0081] The target suppression module uses the target NMS algorithm to eliminate redundant bounding boxes. The target NMS algorithm is obtained by improving the initial NMS algorithm based on the Gaussian weight function. The target NMS algorithm is shown in Formula 2.

[0082] Formula 2 is:

[0083] Among them, N i is the first IOU threshold, N i <N t ,f(IOU(M,b i )) is the Gaussian weight function.

[0084] Gaussian weight function f(IOU(M,b i )) can be calculated based on formula 3.

[0085] Formula 3 is: Among them, σ is the penalty factor.

[0086] The classification module can partially use a fully connected layer and a Softmax function based on the proposed feature map to determine the category of each region, and use proposal box regression to obtain more accurate coordinates of each detection box.

[0087] Step 220: Train the initial foreign object monitoring model to obtain a target foreign object monitoring model.

[0088] In this embodiment, after the processing device constructs an initial foreign object monitoring model based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function, the initial foreign object monitoring model can be trained to obtain a target foreign object monitoring model.

[0089] Specifically, after the processing device constructs the initial foreign object monitoring model based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function, it can obtain the historical substation scene image, and obtain the corresponding sample set based on the historical substation scene image, the sample set includes the initial training set and the test set, and input the initial training set to the initial foreign object monitoring model for training to obtain the target foreign object monitoring model. The target foreign object monitoring model is constructed based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function, which is conducive to improving the monitoring accuracy and efficiency of foreign object monitoring of the substation according to the target foreign object monitoring model.

[0090] Among them, the processing device can annotate the historical substation scene images to obtain corresponding annotated images, and divide the annotated images according to preset division rules to obtain initial training sets and test sets corresponding to the annotated images.

[0091] When the processing device annotates the historical substation scene image, it mainly annotates the type of foreign matter, the location of the foreign matter, and the size of the foreign matter in the historical substation scene image. For example, the processing device can annotate the type of foreign matter in the historical substation scene image with an annotation box, annotate the coordinates of the corner points of the annotation box, and annotate the size of the annotation box.

[0092] The preset division rule may be an artificial rule, for example, the preset division rule may be an artificial rule of initial training set: test set = 8:5, when the sample set contains 13,000 annotated images, the initial training set is 8,000, and the test set is 5,000; the preset division rule may also be an artificial rule of initial training set: test set = 1:1, when the sample set contains 20,000 annotated images, the initial training set is 10,000, and the test set is 10,000. The type of preset division rule is not limited here, and it can be set according to actual needs.

[0093] In some embodiments, after the processing device constructs the initial foreign object monitoring model based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function, it can obtain historical substation scene images, and obtain the corresponding sample set based on the historical substation scene images, and perform data enhancement processing on the initial training set to obtain an enhanced training set, and input the enhanced training set into the initial foreign object monitoring model for training to obtain a target foreign object monitoring model, thereby avoiding the situation where the robustness of the target foreign object monitoring model obtained by training the initial foreign object monitoring model with a smaller initial training set due to the smaller number of historical substation scene images containing foreign object information is low, thereby increasing the robustness of the target foreign object monitoring model.

[0094] Step 230: Acquire the current substation scene image.

[0095] Step 240: Input the current substation scene image into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result.

[0096] In this embodiment, step 230 and step 240 may refer to the contents of the corresponding steps in the aforementioned embodiment, which will not be described in detail here.

[0097] The solution provided in this embodiment constructs an initial foreign object monitoring model based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function, and trains the initial foreign object monitoring model to obtain a target foreign object monitoring model, obtains the current substation scene image, and inputs the current substation scene image into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result, thereby realizing the construction of the target foreign object monitoring model based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function, which is beneficial to improving the monitoring accuracy and efficiency of foreign object monitoring of substations according to the target foreign object monitoring model.

[0098] See also Figure 5 , which shows a flow chart of a foreign body monitoring method provided by another embodiment of the present application. In a specific embodiment, the foreign body monitoring method can be applied to Figure 1 The processing device 300 in the foreign body monitoring system shown in FIG. Figure 5 The process shown is described in detail, and the foreign matter monitoring method may include the following steps 310 and 370.

[0099] Step 310: construct an initial foreign body monitoring model based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function.

[0100] Step 320: Acquire historical substation scene images.

[0101] Step 330: Obtain a corresponding sample set based on the historical substation scene image.

[0102] Step 340: Input the initial training set to the initial foreign body monitoring model for training to obtain the target foreign body monitoring model.

[0103] In this embodiment, step 310, step 320, step 330 and step 340 may refer to the contents of the corresponding steps in the aforementioned embodiment, which will not be repeated here.

[0104] Step 350: Determine whether the target foreign body monitoring model converges according to the test set.

[0105] In this embodiment, the processing device inputs the initial training set to the initial foreign object monitoring model for training. After obtaining the target foreign object monitoring model, the test set can be input to the target foreign object monitoring model. The target foreign object monitoring model receives and responds to the test set, performs testing according to the test set, obtains corresponding test results, and determines the test accuracy corresponding to the test results based on the test results and the test set, and determines whether the target foreign object monitoring model converges based on the test accuracy.

[0106] When the test accuracy is greater than or equal to the preset accuracy threshold, it is determined that the target foreign body monitoring model has converged; when the test accuracy is less than the preset accuracy threshold, it is determined that the target foreign body monitoring model has not converged.

[0107] Among them, the preset accuracy threshold can be used to characterize the minimum accuracy corresponding to the convergence of the target foreign object monitoring model. The preset accuracy threshold can be an accuracy pre-set by the user, or it can be an accuracy automatically generated by the processing equipment according to the monitoring process of multiple foreign object monitoring of the substation. The setting method of the preset accuracy threshold is not limited here, and it can be set according to actual needs.

[0108] Step 360: When it is determined according to the test set that the target foreign object monitoring model converges, the current substation scene image is acquired.

[0109] In this embodiment, when the processing device determines that the target foreign object monitoring model has converged according to the test set, the current substation scene image can be obtained, so that when the target foreign object monitoring model is determined to have converged, foreign object monitoring of the substation is performed based on the target foreign object monitoring model. Since the target foreign object monitoring model has a high stability, it is beneficial to improve the monitoring credibility of foreign object monitoring of the substation according to the target foreign object monitoring model.

[0110] Step 370: Input the current substation scene image into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result.

[0111] In this embodiment, step 370 may refer to the contents of the corresponding steps in the aforementioned embodiments, which will not be described in detail here.

[0112] The solution provided in this embodiment is based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function to construct an initial foreign object monitoring model, obtain historical substation scene images, and obtain corresponding sample sets based on the historical substation scene images, and input the initial training set into the initial foreign object monitoring model for training to obtain a target foreign object monitoring model, and determine whether the target foreign object monitoring model converges based on the test set, and when it is determined that the target foreign object monitoring model converges based on the test set, obtain the current substation scene image, and input the current substation scene image into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result, so that when it is determined that the target foreign object monitoring model converges, the substation is monitored for foreign objects based on the target foreign object monitoring model, and since the target foreign object monitoring model has high stability, it is beneficial to improve the monitoring credibility of foreign object monitoring of the substation based on the target foreign object monitoring model.

[0113] See also Figure 6 , which shows a foreign body monitoring device 400 provided by an embodiment of the present application. The foreign body monitoring device 400 can be applied to Figure 1 The processing device 300 in the foreign body monitoring system shown in FIG. Figure 6 The foreign object monitoring device 400 shown is described in detail. The foreign object monitoring device 400 may include an acquisition module 410 and an input module 420.

[0114] The acquisition module 410 can be used to acquire the current substation scene image; the input module 420 can be used to input the current substation scene image into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result. The target foreign object monitoring model is obtained based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function.

[0115] In some embodiments, the foreign object monitoring device 400 may further include a construction module and a training module.

[0116] The construction module can be used to construct an initial foreign object monitoring model based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function before the acquisition module 410 acquires the current substation scene image; the training module can be used to train the initial foreign object monitoring model to obtain the target foreign object monitoring model.

[0117] In some embodiments, a building block may include a building block and a fusion block.

[0118] The construction unit can be used to construct a target suppression module based on the non-maximum suppression module and the Gaussian weight function; the fusion unit can be used to fuse the region of interest pooling layer and the target suppression module of the Faster R-CNN model to obtain an initial foreign body monitoring model.

[0119] In some implementations, the training module may include a first acquisition unit, a second acquisition unit, and a first input unit.

[0120] The first acquisition unit can be used to acquire historical substation scene images; the second acquisition unit can be used to acquire a corresponding sample set based on the historical substation scene images, and the sample set can at least include an initial training set; the first input unit can be used to input the initial training set into the initial foreign object monitoring model for training to obtain a target foreign object monitoring model.

[0121] In some embodiments, the foreign matter monitoring device 400 may further include a first processing module.

[0122] The first processing module can be used for the first input unit to input the initial training set to the initial foreign body monitoring model for training, and before obtaining the target foreign body monitoring model, perform data enhancement processing on the initial training set to obtain an enhanced training set.

[0123] In some embodiments, the first input unit may include an input sub-unit.

[0124] The input subunit can be used to input the enhanced training set into the initial foreign body monitoring model for training to obtain the target foreign body monitoring model.

[0125] In some embodiments, the sample set may further include a test set, and the foreign matter monitoring device 400 may further include a determination module.

[0126] The determination module may be used to determine whether the target foreign object monitoring model converges according to the test set before the acquisition module 410 acquires the current substation scene image.

[0127] In some implementations, the acquisition module 410 may include a third acquisition unit.

[0128] The third acquisition unit may be configured to acquire the current substation scene image when it is determined according to the test set that the target foreign object monitoring model converges.

[0129] In some embodiments, the foreign matter monitoring device 400 may further include a second processing module.

[0130] The second processing module can be used for the input module 420 to input the current substation scene image to the target foreign object monitoring model, and before obtaining the corresponding foreign object monitoring result, the current substation scene image is subjected to sliding slicing processing to obtain multiple slice images.

[0131] In some embodiments, the input module 420 may include a second input unit.

[0132] The second input unit can be used to input multiple slice images and the current substation scene image into the target foreign object monitoring model to obtain the foreign object monitoring result.

[0133] The solution provided in this embodiment obtains the corresponding foreign object monitoring result by acquiring the current substation scene image and inputting the current substation scene image into the target foreign object monitoring model. The target foreign object monitoring model is obtained based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function. The target foreign object monitoring model constructed based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function is used to monitor the substation for foreign objects, thereby avoiding the influence of the fatigue of the inspectors on manual patrol inspections, resulting in misjudgment of foreign objects in the substation, and improving the monitoring accuracy and efficiency of foreign object monitoring in the substation.

[0134] Furthermore, the non-maximum suppression module is improved based on the Gaussian weight function, so that smaller foreign objects can be easily identified by the target foreign object monitoring model, avoiding the missed detection of smaller foreign objects by the target foreign object monitoring model, which is conducive to further improving the monitoring accuracy of foreign objects in substations.

[0135] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. Any processing method described in the method embodiment can be implemented by the corresponding processing module in the device embodiment, and will not be repeated one by one in the device embodiment.

[0136] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0137] See also Figure 7, which shows a functional block diagram of an electronic device 500 provided by an embodiment of the present application, and the electronic device 500 may include one or more of the following components: a memory 510, a processor 520, and one or more applications, wherein the one or more applications may be stored in the memory 510 and configured to be executed by the one or more processors 520, and the one or more applications are configured to execute the method described in the aforementioned method embodiment.

[0138] The memory 510 may include a random access memory (RAM) or a read-only memory (ROM). The memory 510 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 510 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as obtaining a current substation scene image, inputting a current substation scene image, obtaining a foreign body monitoring result, constructing an initial foreign body monitoring model, training an initial foreign body monitoring model, obtaining a target foreign body monitoring model, constructing a target suppression module, an area of ​​interest pooling layer and a target suppression module, obtaining an initial foreign body monitoring model, obtaining a historical substation scene image, obtaining a sample set, inputting an initial training set, data enhancement processing, obtaining an enhanced training set, inputting an enhanced training set, determining whether the target foreign body monitoring model converges, determining whether the target foreign body monitoring model converges, sliding slice processing, obtaining multiple slice images, and inputting multiple slice images and the current substation scene image, etc.), instructions for implementing the following various method embodiments, etc. The data storage area can also store data created by the electronic device 500 during use (such as the current substation scene image, target foreign object monitoring model, foreign object monitoring results, Faster R-CNN model, non-maximum suppression module, Gaussian weight function, initial foreign object monitoring model, target suppression module, region of interest pooling layer, historical substation scene images, sample set, initial training set, enhanced training set, test set and multiple slice images), etc.

[0139] The processor 520 may include one or more processing cores. The processor 520 uses various interfaces and lines to connect various parts of the entire electronic device 500, and executes various functions and processes data of the electronic device 500 by running or executing instructions, programs, code sets or instruction sets stored in the memory 510, and calling data stored in the memory 510. Optionally, the processor 520 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), programmable logic array (PLA), etc. The processor 520 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 520, but may be implemented separately through a communication chip.

[0140] Please refer to Figure 8 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 600 stores a program code 610, which can be called by a processor to execute the method described in the above method embodiment.

[0141] The computer readable storage medium 600 can be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 600 includes a non-transitory computer-readable storage medium. The computer readable storage medium 600 has storage space for program code 610 that performs any method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code 610 can be compressed, for example, in an appropriate form.

[0142] Please refer to Fig. 9, which shows a structural block diagram of a computer program product 700 provided in an embodiment of the present application. The computer program product 700 includes a computer program / instruction 710, and the computer program / instruction 710 is stored in a computer-readable storage medium of a computer device. When the computer program product 700 is run on a computer device, a processor of the computer device reads the computer program / instruction 710 from the computer-readable storage medium, and the processor executes the computer program / instruction 710, so that the computer device performs the method described in the above method embodiment.

[0143] The solution provided in this embodiment obtains the corresponding foreign object monitoring result by acquiring the current substation scene image and inputting the current substation scene image into the target foreign object monitoring model. The target foreign object monitoring model is obtained based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function. The target foreign object monitoring model constructed based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function is used to monitor the substation for foreign objects, thereby avoiding the influence of the fatigue of the inspectors on manual patrol inspections, resulting in misjudgment of foreign objects in the substation, and improving the monitoring accuracy and efficiency of foreign object monitoring in the substation.

[0144] Furthermore, the non-maximum suppression module is improved based on the Gaussian weight function, so that smaller foreign objects can be easily identified by the target foreign object monitoring model, avoiding the missed detection of smaller foreign objects by the target foreign object monitoring model, which is conducive to further improving the monitoring accuracy of foreign objects in substations.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A foreign body monitoring method, characterized in that: include: Get the current substation scene image; The current substation scene image is input into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result. The target foreign object monitoring model is obtained based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function.

2. The foreign body monitoring method according to claim 1, characterized in that: Before obtaining the current substation scene image, the foreign matter monitoring method further includes: Based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function, an initial foreign body monitoring model is constructed; The initial foreign body monitoring model is trained to obtain the target foreign body monitoring model.

3. The foreign body monitoring method according to claim 2, characterized in that: The initial foreign body monitoring model is constructed based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function, including: Based on the non-maximum suppression module and the Gaussian weight function, construct a target suppression module; The region of interest pooling layer of the Faster R-CNN model and the target suppression module are integrated to obtain the initial foreign body monitoring model.

4. The foreign body monitoring method according to claim 2, characterized in that: The training of the initial foreign body monitoring model to obtain the target foreign body monitoring model includes: Obtain historical substation scene images; According to the historical substation scene image, a corresponding sample set is obtained, wherein the sample set at least includes an initial training set; The initial training set is input into the initial foreign body monitoring model for training to obtain the target foreign body monitoring model.

5. The foreign matter monitoring method according to claim 4, characterized in that: Before inputting the initial training set into the initial foreign body monitoring model for training to obtain the target foreign body monitoring model, the foreign body monitoring method further includes: Performing data enhancement processing on the initial training set to obtain an enhanced training set; The inputting the initial training set into the initial foreign body monitoring model for training to obtain the target foreign body monitoring model comprises: The enhanced training set is input into the initial foreign body monitoring model for training to obtain the target foreign body monitoring model.

6. The foreign matter monitoring method according to claim 4, characterized in that: The sample set also includes a test set. Before acquiring the current substation scene image, the foreign object monitoring method also includes: Determining whether the target foreign body monitoring model converges according to the test set; The obtaining of the current substation scene image includes: When it is determined according to the test set that the target foreign object monitoring model converges, the current substation scene image is acquired.

7. The foreign matter monitoring method according to any one of claims 1 to 6, characterized in that: Before inputting the current substation scene image into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result, the foreign object monitoring method further includes: Performing sliding slicing processing on the current substation scene image to obtain multiple slice images; The inputting the current substation scene image into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result includes: The multiple slice images and the current substation scene image are input into the target foreign object monitoring model to obtain the foreign object monitoring result.

8. A foreign body monitoring device, characterized in that: include: An acquisition module is used to acquire the current substation scene image; The input module is used to input the current substation scene image into the target foreign object monitoring model to obtain the corresponding foreign object monitoring result. The target foreign object monitoring model is obtained based on the Faster R-CNN model, the non-maximum suppression module and the Gaussian weight function.

9. An electronic device, characterized in that: include: Memory; One or more processors coupled to the memory; One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by one or more processors, and the one or more applications are configured to execute the foreign matter monitoring method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the foreign matter monitoring method according to any one of claims 1 to 7.

Images