Method and system for monitoring abnormal intrusion of power equipment indoors and outdoors and storage medium

Abstract

The application relates to the technical field of power equipment safety, and discloses a power equipment indoor and outdoor abnormal intrusion monitoring method and system and a storage medium. The method comprises the following steps: photographing power equipment through at least two cameras to obtain a target image; comparing the target image with a historically stored normal image to obtain an abnormal pixel point and the type of a target power equipment to which the abnormal pixel point belongs; performing position clustering on the abnormal pixel point to obtain the dispersion and coverage of the abnormal pixel point; inputting the type of the target power equipment, the dispersion, the coverage, the electrical parameter, the vibration parameter and the voiceprint feature of the target power equipment into a neural network model to obtain an intrusion type and an intrusion degree. The embodiment can accurately monitor the natural disaster type of abnormal intrusion through an intelligent method.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power equipment safety, and in particular to a power equipment indoor and outdoor abnormal intrusion monitoring method and system and a storage medium. BACKGROUND

[0002] Existing power equipment includes power transmission equipment, power distribution equipment, power generation equipment and energy storage equipment, etc. When these power equipment are subjected to natural disasters such as flowing water, sand and strong wind, they may be damaged, the foundation may be unstable, and even collapse. Although some power equipment is arranged indoors, it may also be affected by natural disasters.

[0003] In the prior art, in order to avoid the influence of indoor and outdoor power equipment subjected to natural disasters, preventive measures are mostly taken, such as reinforcing the barrier before the disaster arrives and increasing personnel on duty. However, this way generally increases the intrusion monitoring cost and causes resource waste.

[0004] In the prior art, there is a lack of timely and accurate early warning strategy for power equipment to give early warning of possible influence on power equipment. SUMMARY

[0005] In order to solve the above technical problems, the present application provides a power equipment indoor and outdoor abnormal intrusion monitoring method and system and a storage medium to accurately monitor the abnormal intrusion of natural disasters by an intelligent method.

[0006] In a first aspect, the present application provides a power equipment indoor and outdoor abnormal intrusion monitoring method, comprising:

[0007] photographing the power equipment by at least two cameras to obtain a target image;

[0008] comparing the target image with a historically stored normal image to obtain an abnormal pixel point and a type of target power equipment to which the abnormal pixel point belongs;

[0009] position clustering the abnormal pixel point to obtain a dispersion degree and a coverage degree of the abnormal pixel point;

[0010] inputting the type of target power equipment, dispersion degree, coverage degree, and electrical parameters, vibration parameters and voiceprint features of the target power equipment into a neural network model to obtain an intrusion type and an intrusion degree.

[0011] In a second aspect, the present application provides a power equipment indoor and outdoor abnormal intrusion monitoring system, comprising:

[0012] at least two cameras and a processor;

[0013] The processor is used for executing the power equipment indoor and outdoor abnormal intrusion monitoring method.

[0014] In a third aspect, an embodiment of the present application provides a computer readable storage medium, and the medium stores computer instructions, and the computer instructions are used for making the computer execute the power equipment indoor and outdoor abnormal intrusion monitoring method.

[0015] The embodiment of the present application has the following technical effects:

[0016] 1. The present application can analyze the intrusion type and the intrusion degree of natural disasters through the target images photographed by at least two cameras, without the parameters of other types of sensors, and has low cost.

[0017] 2. Since the power equipment generally has a fixed position and appearance, the present application can directly use the pixel point comparison method in the image to determine the abnormal pixel points, without complex image recognition. The dispersion and the coverage can be obtained through the clustering of the abnormal pixel points, and the visual performance characteristics of the natural disasters can be known. In combination with the performance of the power equipment, such as the electrical parameters, the vibration parameters and the voiceprint characteristics, and in combination with the type of the target power equipment, the intrusion type and the intrusion degree suitable for the type, performance and visual characteristics of the target power equipment can be analyzed by using the powerful learning ability of the neural network, so that the accuracy of the intrusion detection is improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0019] Figure 1 is a flowchart of a power equipment indoor and outdoor abnormal intrusion monitoring method provided by an embodiment of the present application;

[0020] Figure 2 is a structural schematic diagram of a neural network model provided by an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present application more clear, the technical solutions of the present application will be clearly and completely described below. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of the present application.

[0022] The method provided by the embodiment is suitable for identifying the type and degree of invasion of natural disasters, such as water flow, sand or mud covering the power equipment. The embodiment does not consider the invasion of people or animals, which can be monitored by an infrared sensor. In a specific implementation, the invasion of people or animals is excluded by the infrared sensor first, and then the method provided by the embodiment is used when no invasion of people or animals is detected.

[0023] Figure 1 is a flowchart of a method for monitoring abnormal invasion indoors and outdoors of power equipment provided by an embodiment of the present application, Figure 2 is a structural diagram of a neural network model provided by an embodiment of the present application.

[0024] Referring to Figure 1 and Figure 2 the method provided by the embodiment includes the following operations:

[0025] S110, photographing the power equipment by at least two cameras to obtain a target image.

[0026] The at least two cameras can irradiate different positions of the power equipment body and the surrounding environment to obtain a panoramic image including the power equipment.

[0027] Optionally, a near-field image is obtained by photographing the near-field environment of the power equipment by at least one near-field camera, and a far-field image is obtained by photographing the far-field environment of the power equipment by at least one far-field camera. The near-field environment of the power equipment can be an environment within 50 meters from the power equipment, and the far-field environment of the power equipment can be an environment within 200 meters from the power equipment. The near-field environment and the far-field environment can be selected according to the performance of the camera.

[0028] Then, the near-field image and the far-field image are fused to obtain the target image. The target image obtained in this way comprehensively reflects the overall situation of the environment around the power equipment, and has the advantages of high near-field definition and wide far-field field of view.

[0029] The embodiment collects near-field images and far-field images in real time, and fuses the target image after time stamp alignment. The target image reflects whether the power equipment is currently being invaded by natural disasters. The normal image that has not received the invasion of natural disasters is stored in the local database. First, the light and contrast of the near-field image and the far-field image are adjusted according to the normal image, so that the near-field image and the far-field image have the same exposure and contrast as the normal image, avoiding the influence of light and shadow. Then, the near-field image and the far-field image are synthesized according to the same physical position to obtain the target image. For example, based on the conversion relationship between the camera coordinate system and the earth coordinate system, the near-field image and the far-field image are projected into the earth coordinate system. Similarly, the normal image is also projected into the earth coordinate system. In the earth coordinate system, the image pixels corresponding to the same coordinate points are synthesized, for example, the pixel values are averaged to obtain the pixel value of the synthesized pixel point.

[0030] The embodiment synthesizes the near-field image and the far-field image to obtain a target image with a wide and clear field of view.

[0031] S120, comparing the target image with the historically stored normal image to obtain abnormal pixel points and the type of target power equipment to which the abnormal pixel points belong.

[0032] First, the power equipment is divided into a first part with fixed position and a second part with floating position. For example, the power equipment is a power storage device, the first part with fixed position is a battery box, and the second part with floating position is a cable connected to the battery box. The cable may change its position due to external forces such as human force and wind force. When the cable can still work normally after changing its position, the new position is also an allowed position, that is, the external force at this time cannot be considered as the invasion of natural disasters. In order to avoid misjudgment, the position of the second part includes the position of the second part when it is subjected to external force and works normally, that is, the original position of the cable is expanded.

[0033] In the target image, the pixel points that do not conform to the positions of the first part and the second part are determined as abnormal pixel points. Specifically, in the positions of the first part / second part of the target image and the normal image, the pixel values are compared pixel by pixel, and the pixel points with an average RGB three-channel pixel difference value exceeding a set threshold value are determined as abnormal pixel points. For example, there is a water stain on the battery, and the pixel points at the water stain are abnormal pixel points.

[0034] The power equipment to which the abnormal pixel points belong is called target power equipment, for example, the type is battery.

[0035] S130, clustering the positions of the abnormal pixel points to obtain the dispersion and coverage of the abnormal pixel points.

[0036] For example, the K-Means clustering method is used to perform distance-based clustering on the abnormal pixel points, and a cluster is obtained on the target image. The clusters are separated from each other, and each cluster has a center point.

[0037] In a cluster, the distance of each abnormal pixel point from the cluster center is calculated, and the size of each cluster is calculated according to the distance. For example, a cluster has 30 abnormal pixel points, the distance of 29 pixel points other than the center from the center is calculated, and the average distance is obtained to obtain the size of each cluster. The number of all clusters is divided by the average size of all clusters to obtain the dispersion of all abnormal pixel points, that is, a dispersion is obtained for the target image. The more the number of clusters and the smaller the average size, the more dispersed the distribution of the clusters. The material with high water content has continuity, for example, sewage, and the dispersion is low; the material with low water content does not have continuity, for example, sand, and the dispersion is high.

[0038] The number of pixel points contained in all clusters is calculated, and the proportion of the number of pixel points in the total number of pixel points of the target image is taken as the coverage. For example, 5 clusters are clustered on the target image, and there are 1500 pixel points in the 5 clusters. The total number of pixel points of the target image is 3000, and the coverage is 50%. Obviously, the higher the coverage, the higher the degree of intrusion.

[0039] S140, inputting the type, dispersion, coverage, and electrical parameters, vibration parameters, and voiceprint features of the target power equipment into the neural network model to obtain the intrusion type and the intrusion degree.

[0040] The electrical parameters of the target power equipment include but are not limited to voltage, current, and power, etc., which can be obtained by real-time monitoring by a power meter. The vibration parameters include the vibration amplitude and vibration frequency of the surface of the target power equipment (such as the surface of the battery box), which can be obtained by collecting the signals of the pressure sensor installed on the shell of the power equipment. The voiceprint features include the voiceprint features in the environment of the target power equipment, such as wind sound and sandstone hitting sound, which can be obtained by collecting the signals of the microphone installed in the environment of the power equipment. When the target power equipment is subjected to different types of intrusion and different degrees of intrusion, the electrical parameters, vibration parameters, and voiceprint features are also different, which can be used as input features of the neural network model.

[0041] Referring to Figure 2 , the neural network model includes an embedding layer, an attention layer, a feature extraction layer, and a classification layer.

[0042] Firstly, the type, dispersion, coverage, electrical parameter, vibration parameter and voiceprint feature of the target power equipment are input to the embedding layer to obtain the embedding representation (essentially a feature vector) of each input parameter. The embedding layer can map discrete data to continuous data, which is convenient for neural network processing. The embedding representation is automatically optimized during the training process and does not need to be manually designed. It should be noted that if the voiceprint feature is already in the form of a feature vector, it can be directly input to the attention layer without passing through the embedding layer.

[0043] The embedding representations of the dispersion and coverage are spliced (for example, spliced in front and back), and the spliced vector comprehensively represents the visual performance characteristics of the target power equipment.

[0044] Then, the spliced vector is input to the attention layer together with the embedding representation of the electrical parameter, the embedding representation of the vibration parameter and the embedding representation of the voiceprint feature to obtain a first vector of the spliced vector mapped to the electrical parameter, a second vector of the spliced vector mapped to the vibration parameter, and a third vector of the spliced vector mapped to the voiceprint feature. The attention layer is a dynamic weight allocation mechanism that can adaptively focus on the importance of the output according to different parts of the input data. Its core idea is to simulate the human attention mechanism, that is, "ignore irrelevant information and focus on key parts". Three attention modules are designed in the attention layer in this embodiment, the spliced vector and the embedding representation of the electrical parameter are input to the first attention module to obtain the first vector, the spliced vector and the embedding representation of the vibration parameter are input to the second attention module to obtain the second vector, and the spliced vector and the embedding representation of the voiceprint feature are input to the third attention module to obtain the third vector. Each attention module can be a Transformer encoder. The embodiment can extract the performance characteristics of the electrical parameter, the vibration parameter and the voiceprint feature from the visual performance characteristics through the attention module, which is conducive to improving the accuracy of the final recognition.

[0045] Next, the first vector, the second vector, the third vector and the embedding representation of the type are input to the feature extraction layer to obtain intermediate features. The feature extraction layer is used for feature fusion. The intermediate features are input to the classification layer to obtain the intrusion type and the intrusion degree. The classification layer outputs a feature of 2*n dimensions, where n represents the total number of intrusion types and the dimension number. Each type / dimension includes a confidence and an intrusion degree (represented by a percentage). If the confidence is greater than 90%, it is considered that the intrusion type is recognized and the corresponding intrusion degree is obtained. For example, the feature vector of one dimension (corresponding to the sandstone type) output by the classification layer is 0.91, 0.6, indicating that the intrusion type is sandstone and the intrusion degree is 60% of the power equipment.

[0046] The embodiment of the present application has the following technical effects:

[0047] 1. The application can analyze the invasion type and the invasion degree of natural disasters through at least two camera shots, without the need for other types of sensor parameters, and has low cost.

[0048] 2. Since power equipment generally has a fixed position and appearance, the application can directly use the pixel point comparison method in the image to determine the abnormal pixel points, without the need for complex image recognition. The dispersion and coverage obtained through clustering of the abnormal pixel points can know the visual performance characteristics of natural disasters; combined with the performance of the power equipment itself: electrical parameters, vibration parameters and voiceprint characteristics; combined with the type of the target power equipment, the powerful learning ability of the neural network is used to analyze the invasion type and the invasion degree suitable for the type, performance and visual characteristics of the target power equipment, thereby improving the accuracy of the intrusion detection.

[0049] Before using the neural network model to identify the invasion type and the invasion degree, the neural network model needs to be trained using training samples. Specifically, positive image samples and negative image samples of power equipment occurring indoor and outdoor abnormal invasions of different degrees at historical times are collected; for the positive image samples, a sample expansion technique is used to generate new positive image samples; the neural network model is trained using the positive image samples and the negative image samples, as well as the invasion type label and the invasion degree label. The sample expansion technique aims to increase the diversity of training data by generating new samples or modifying existing samples to improve model performance, such as data enhancement techniques, transfer learning methods and self-supervised learning methods.

[0050] The embodiment of the application also provides an indoor and outdoor abnormal intrusion monitoring system for power equipment, which comprises at least two cameras and a processor. The camera is electrically connected with the processor and sends the photographed image to the processor. The processor is used to execute the indoor and outdoor abnormal intrusion monitoring method for power equipment.

[0051] The embodiment of the application also provides a computer readable storage medium. The medium stores computer instructions for making the computer execute the above method. The computer instructions on the computer readable storage medium are used to make the computer execute the above method, and thus have at least the same advantages as the above method.

[0052] The medium in the present application can adopt any combination of one or more computer-readable media. The medium can be a computer-readable signal medium or a computer-readable storage medium. The medium may, for example, but is not limited to, an electrical, a magnetic, an optical, an electromagnetic, an infrared, or a semiconductor system, device or apparatus, or any appropriate combination thereof. More specific examples (a non-exhaustive list) of the medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In this document, the medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device.

[0053] The computer-readable signal medium can include a computer-readable program code in a baseband or propagated as a carrier wave in a propagation medium. Such a propagated signal can take a wide variety of forms, including but not limited to, electro-magnetic, optical, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium that is not a computer-readable storage medium and that can communicate programs for use by or in connection with an instruction execution system, apparatus, or device.

[0054] Program code embodied on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wire line, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0055] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In an embodiment, the present application is directed to computer program products comprising machine-readable media for carrying or having computer readable program code embodied therein.

[0056] It should be noted that the terms used in the present application are only intended to describe specific embodiments and are not intended to limit the scope of the present application. As shown in the specification of the present application, unless the context clearly indicates otherwise, the words "one", "a", "an", and / or "the" do not specifically refer to the singular, but can also include the plural. The terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method or device including the element.

[0057] It should also be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only used to facilitate the description of the present application and simplify the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. Unless otherwise specified and limited, the terms "mounting", "connection", "connection" and the like should be broadly understood, for example, it can be a fixed connection, or it can be a detachable connection, or an integral connection; it can be a mechanical connection, or an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not limited thereto; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present application.

Claims

1. A method for monitoring abnormal intrusions into power equipment indoors and outdoors, characterized in that, include: The target image is obtained by capturing images of the electrical equipment using at least two cameras; The target image is compared with historically stored normal images to identify power equipment objects, thereby obtaining abnormal pixels and the type of target power equipment to which the abnormal pixels belong. The abnormal pixels are clustered by location to obtain their dispersion and coverage. The type, dispersion, and coverage of the target power equipment, along with its electrical parameters, vibration parameters, and acoustic signature features, are input into a neural network model to obtain the intrusion type and intrusion degree, including: The type, dispersion, coverage, electrical parameters, vibration parameters, and acoustic signature features of the target power equipment are input into the embedding layer to obtain the embedded representation of each input parameter. The embedded representations of dispersion and coverage are concatenated and then input into the attention layer along with the embedded representations of electrical parameters, vibration parameters, and acoustic signature features to obtain a first vector mapped to the electrical parameters, a second vector mapped to the vibration parameters, and a third vector mapped to the acoustic signature features. The first, second, and third vectors, along with the type's embedded representation, are input into the feature extraction layer to obtain intermediate features. The intermediate features are then input into the classification layer to obtain the intrusion type and intrusion degree. The neural network model includes an embedding layer, an attention layer, a feature extraction layer, and a classification layer.

2. The method for monitoring indoor and outdoor abnormal intrusions into power equipment according to claim 1, characterized in that, Images of the target equipment are obtained by capturing images using at least two cameras, including: Near-field images are obtained by capturing images of the near-field environment of the power equipment using at least one near-field camera; A far-field image is obtained by capturing images of the remote environment of the power equipment using at least one far-field camera; The near-field image and the far-field image are fused to obtain the target image.

3. The method for monitoring indoor and outdoor abnormal intrusions into power equipment according to claim 2, characterized in that, The near-field image and the far-field image are fused to obtain the target image, including: Based on the normal image, adjust the brightness and contrast of the near-field and far-field images; The target image is obtained by combining the near-field image and the far-field image at the same physical location.

4. The method for monitoring indoor and outdoor abnormal intrusions into power equipment according to claim 3, characterized in that, The target image is compared with historically stored normal images of power equipment objects to obtain abnormal pixels, including: The power equipment is divided into a first part with a fixed position and a second part with a floating position. Pixels in the target image that do not conform to the positions of the first part and the second part are identified as abnormal pixels. The second part includes at least a cable, and the location of the second part includes the position of the second part when it is subjected to external force and is operating normally.

5. The method for monitoring indoor and outdoor abnormal intrusions into power equipment according to claim 4, characterized in that, The abnormal pixels are clustered by location to obtain their dispersion and coverage, including: Within a cluster, calculate the distance of each anomalous pixel from the cluster center, and calculate the size of each cluster based on the distance; divide the number of all clusters by the average size of all clusters to obtain the dispersion of all anomalous pixels; Calculate the number of pixels contained in all clusters, and use the proportion of the number of pixels in the clusters to the total number of pixels in the target image as the coverage.

6. The method for monitoring indoor and outdoor abnormal intrusions into power equipment according to claim 1, characterized in that, Before obtaining the target image by photographing the power equipment using at least two cameras, the process also includes: Collect positive image samples when abnormal intrusions of power equipment occur at different levels indoors and outdoors, and negative image samples when no abnormal intrusions occur; For the positive image sample, a new positive image sample is generated using sample dilation technology; The neural network model is trained using the positive and negative image samples, as well as the intrusion type and intrusion degree labels.

7. A power equipment indoor / outdoor abnormal intrusion monitoring system, characterized in that, include: At least two cameras and a processor; The processor is used to execute the indoor and outdoor abnormal intrusion monitoring method for power equipment as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The medium stores computer instructions, which are used to cause the computer to execute the indoor and outdoor abnormal intrusion monitoring method for power equipment as described in any one of claims 1-6.

Images