Electrical equipment change detection method and device, electronic equipment and readable storage medium

Abstract

The invention is suitable for the technical field of transformer substations, and provides an electrical equipment change detection method and device, electronic equipment and a readable storage medium. The method comprises the steps that to-be-detected image pairs of the same first to-be-detected area in the transformer substation at different times are acquired, the to-be-detected image pairs comprise a first to-be-detected image and a second to-be-detected image, and the shooting time of the second to-be-detected image is later than that of the first to-be-detected image; inputting the to-be-detected image pair into an electrical equipment change detection network to obtain an electrical equipment change detection result; if the electrical equipment change detection result indicates that the electrical equipment changes, generating a detection frame for marking a target change area in the second to-be-detected image based on center point information and width and height information of the target change area in the electrical equipment change detection result, the target change area is an area where the electrical equipment changes relative to the first to-be-detected image in the second to-be-detected image. According to the invention, the change detection efficiency of the electrical equipment can be improved.

Description

Technical Field

[0001] The present application belongs to the technical field of substations, and in particular, relates to a method, device, electronic device and readable storage medium for detecting changes in electrical equipment. Background Art

[0002] Change detection of electrical equipment in substations can be used as an auxiliary means of substation inspection. Changes in electrical equipment mainly include damage to meter dials, changes in indicator light status, changes in respirator color, missing components, etc., which are an important part of substation inspections. Using change detection methods to assist substation inspections can improve the efficiency of inspections and thus improve the stability of the entire substation operation.

[0003] Existing change detection methods use pixel-level segmentation of images as the final result. When implementing change detection, each pixel needs to be classified, which is highly complex and leads to low change detection efficiency. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device and readable storage medium for detecting changes in electrical equipment, which can improve the efficiency of detecting changes in electrical equipment.

[0005] In a first aspect, an embodiment of the present application provides a method for detecting changes in an electrical device, comprising:

[0006] Acquire a pair of images to be tested of the same first area to be tested in the substation at different times, the pair of images to be tested comprising a first image to be tested and a second image to be tested, wherein the shooting time of the second image to be tested is later than the shooting time of the first image to be tested;

[0007] Inputting the image pair to be tested into an electrical device change detection network to obtain an electrical device change detection result of the second image to be tested relative to the first image to be tested;

[0008] If the electrical equipment change detection result indicates that an electrical equipment change has occurred, a detection frame for marking the target change area is generated in the second image to be tested based on the center point information and width and height information of the target change area in the electrical equipment change detection result, and the target change area is the area in the second image to be tested where the electrical equipment change has occurred relative to the first image to be tested.

[0009] In an embodiment of the present application, by obtaining a pair of images to be tested of the same first area to be tested in the substation at different times, the pair of images to be tested includes a first image to be tested and a second image to be tested, the shooting time of the second image to be tested is later than the shooting time of the first image to be tested, and the pair of images to be tested is input into the electrical equipment change detection network, a corresponding electrical equipment change detection result can be obtained, if the electrical equipment change detection result indicates that the electrical equipment to be tested has changed, then based on the center point information and width and height information of the target change area in the electrical equipment change detection result, a detection frame for marking the target change area can be generated in the second image to be tested, the target change area is the area of ​​the second image to be tested relative to the electrical equipment change in the first image to be tested. When the electrical equipment change detection network in this scheme detects that an electrical equipment change has occurred, the center point information and width and height information of the change area are output, and a detection frame can be generated based on the center point information and width and height information of the change area, and the change area can be marked in the second image to be tested based on the detection frame without fine pixel-level segmentation of the second image to be tested, thereby reducing the complexity of electrical equipment change detection and improving the efficiency of electrical equipment change detection.

[0010] In a second aspect, an embodiment of the present application provides an electrical equipment change detection device, comprising:

[0011] An image acquisition module, used to acquire a pair of images to be tested of the same first area to be tested in the substation at different times, wherein the pair of images to be tested includes a first image to be tested and a second image to be tested, and the shooting time of the second image to be tested is later than the shooting time of the first image to be tested;

[0012] a change detection module, configured to input the image pair to be tested into an electrical device change detection network, and obtain an electrical device change detection result of the second image to be tested relative to the first image to be tested;

[0013] A detection frame generation module is used to generate a detection frame for marking the target change area in the second image to be tested if the electrical equipment change detection result indicates that the electrical equipment change has occurred, based on the center point information and width and height information of the target change area in the electrical equipment change detection result, wherein the target change area is the area in the second image to be tested where the electrical equipment change has occurred relative to the first image to be tested.

[0014] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect above is implemented.

[0016] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on an electronic device, the electronic device executes the method described in the first aspect above.

[0017] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 It is a flowchart of the method for detecting changes in electrical equipment provided in an embodiment of the present application;

[0020] Figure 2 is a schematic diagram of the structure of an electrical equipment change detection network provided in an embodiment of the present application;

[0021] Figure 3 is a schematic diagram of a training process of an electrical equipment change detection network provided in an embodiment of the present application;

[0022] Figure 4 is a structural diagram of an electrical equipment change detection device provided in an embodiment of the present application;

[0023] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0025] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0026] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0027] It should also be understood that in the embodiments of the present application, "plurality" refers to two or more than two.

[0028] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0029] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0030] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0031] The electrical device change detection method provided in the embodiment of the present application can be applied to electronic devices such as mobile phones, tablet computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), etc. The embodiment of the present application does not impose any restrictions on the specific types of electronic devices.

[0032] See also Figure 1 , Figure 1 The flowchart of the method for detecting changes in electrical equipment provided in the embodiment of the present application is shown. As an example but not a limitation, the method includes the following steps:

[0033] Step 101: acquiring a pair of images to be tested of the same first area to be tested in a substation at different times.

[0034] The image pair to be tested includes a first image to be tested and a second image to be tested, and the shooting time of the second image to be tested is later than the shooting time of the first image to be tested.

[0035] The first area to be tested may refer to any area in the substation, or may refer to an area in the substation where electrical equipment is installed, which is not limited in the present application.

[0036] The electrical equipment in this application may refer to equipment in the substation that requires change detection, including but not limited to primary equipment such as generators and transformers in the substation, secondary equipment such as cabinet doors and meter dials, and components such as indicator lights, breathing lights, and pressure plates.

[0037] It should be noted that the present application does not limit the time interval between the first image to be tested and the second image to be tested. For example, the second image to be tested may be an image of the first area to be tested taken at the current moment, and the first image to be tested may be an image of the first area to be tested taken five months ago.

[0038] In the embodiment of the present application, by taking the first image to be tested as the reference image, it is possible to detect whether the second image to be tested has undergone an electrical device change relative to the first image to be tested, thereby achieving electrical device change detection.

[0039] The above-mentioned changes in electrical equipment include but are not limited to the addition of new electrical equipment, loss of electrical equipment, change in the status of electrical equipment, change in the color of electrical equipment, damage to electrical equipment, etc.

[0040] Before executing step 102, the image pair to be tested may be subjected to noise reduction and smoothing processing to reduce interference of various noises on the detection result of the change of the electrical device. For example, the noise reduction and smoothing processing may be achieved by performing mean filtering processing on the image pair to be tested.

[0041] Step 102: input the image pair to be tested into an electrical device change detection network to obtain an electrical device change detection result of the second image to be tested relative to the first image to be tested.

[0042] The electrical device change detection network is used to detect whether an electrical device change occurs in the second image to be tested relative to the first image to be tested.

[0043] Detecting whether electrical equipment changes occur in the second image to be tested relative to the first image to be tested can be understood as detecting whether electrical equipment changes occur in the first area to be tested relative to the shooting time of the second image to be tested relative to the shooting time of the first image to be tested.

[0044] In an embodiment of the present application, before executing step 102, the electrical equipment change detection network can be trained so that the electrical equipment change detection network can adapt to the task of electrical equipment change detection, improve the generalization ability of the electrical equipment change detection network, and enhance the adaptability of the electrical equipment change detection network.

[0045] After the electrical equipment change detection network is trained, the image pair to be tested is input into the trained electrical equipment change detection network, so that the electrical equipment change detection processing is performed on the image pair to be tested through the trained electrical equipment changes. In a possible implementation, the electrical equipment change detection network includes a target processing layer, a calculation module, a target convolution layer, and a prediction head of a CenterNet target detection network in sequence, and the target processing layer includes a CNN backbone network of a BIT_CD change detection network and a bi-temporal image converter in sequence.

[0046] The above-mentioned electrical equipment change detection network is an improved change detection network. This embodiment is based on the commonly used BIT_CD change detection network, changes the post-processing layer of the network, and combines it with the post-processing layer of the CenterNet target detection network to build an improved change detection network. The improved change detection network uses some layers of the ResNet of the BIT_CD change detection network as feature extractors, divides the obtained feature map into blocks through encoding, and then sends it to the Transformer encoder and Transformer decoder for feature enhancement. Finally, the prediction head of CenterNet is used for model prediction to realize electrical equipment change detection.

[0047] In a possible implementation, the image pair to be tested is input into an electrical device change detection network to obtain an electrical device change detection result, including:

[0048] Inputting the image pair to be tested into the target processing layer to obtain a first feature map of the first image to be tested and a second feature map of the second image to be tested;

[0049] Triggering the calculation module to calculate the absolute value of the difference between the first characteristic map and the second characteristic map to obtain a third characteristic map;

[0050] The third feature map is input into the target convolution layer to obtain a fourth feature map, and the size and number of channels of the fourth feature map match those of the prediction head;

[0051] The fourth feature map is input into the prediction head to perform electrical equipment change detection to obtain an electrical equipment change detection result.

[0052] Among them, the above-mentioned target convolution layer is used to achieve the matching of the output of the target processing layer with the prediction head of the CenterNet target detection network.

[0053] By calculating the absolute value of the difference between the first characteristic map and the second characteristic map, the difference between the second image to be tested and the first image to be tested can be calculated. On this basis, it can be determined whether the second image to be tested has electrical equipment changes relative to the first image to be tested.

[0054] In a possible implementation, the prediction head includes a first convolution layer, a second convolution layer, a third convolution layer, and a generation module; the fourth feature map is input into the prediction head to perform electrical device change detection, and an electrical device change detection result is obtained, including:

[0055] Input the fourth feature map into the first convolutional layer to obtain a heat map of the electrical device change, and when the heat map indicates that the electrical device change has occurred, determine the position information of the center point of the target change area in the fourth feature map from the heat map;

[0056] Input the fourth feature map into the second convolutional layer to obtain the center point offset of the target change area;

[0057] Input the fourth feature map into the third convolutional layer to obtain the width and height information of the target change area;

[0058] The generation module is triggered to generate an electrical equipment change detection result based on the position information of the center point of the target change area in the fourth feature map, the center point offset of the target change area, and the width and height information of the target change area.

[0059] Since downsampling is required to increase the receptive field when generating the feature map, in order to make the prediction result of the prediction head more accurate, the position information of the center point of the target change area can be mapped from the fourth feature map to the second image to be tested based on the center point offset. On this basis, combined with the width and height information of the target change area, a detection frame surrounding the target change area can be generated.

[0060] like Figure 2The figure shows the structure diagram of the electrical equipment change detection network. The first image to be tested and the second image to be tested are passed through the target processing layer (i.e., the CNN backbone network and the dual-temporal image converter of the BIT_CD change detection network) to obtain the first feature map and the second feature map. The absolute value of the difference between the two feature maps is taken to obtain the third feature map reflecting the image change part. The third feature map is further processed. Since the size and number of channels of the third feature map can be 128×128×32, which does not match the size and number of channels of the required feature map of the network post-processing layer of CenterNet, a two-dimensional convolution (i.e., the target convolution layer) is added to adjust the size and number of channels of the third feature map to 128×128×64. The prediction head performs three convolutions on the feature maps after the above adjustment channels. First, the first convolution layer is used. The number of channels of the convolution is the number of target types. In change detection, the object type is to judge whether the area has changed or not. The purpose is to predict whether there is a change area for each thermal point. Then the second and third convolution layers are used. The number of channels of the second convolution layer can be 2. The purpose is to calculate the offset between the center point of each change area and the thermal point. The number of channels of the third convolution layer can be 2. The purpose is to predict the width and height of the change area. Finally, the generation module is used to generate a detection frame (i.e., a detection box for marking the target change area) in the second image to be tested based on the output information of the first, second and third convolution layers. Figure 2 The dotted box in the image output by the generation module).

[0061] In a possible implementation, the training process of the electrical device change detection network includes the following steps 301 to 304 .

[0062] Step 301, obtaining multiple background image pairs.

[0063] Each background image pair includes a first background image and a second background image of the same second test area at different times, and the shooting time of the second background image is later than the shooting time of the corresponding first background image. The second test areas of different background image pairs are different.

[0064] It should be noted that the second test area can be located in the substation or in a non-substation natural environment. Since the environment of the substation is changeable, if multiple different areas are selected from the non-substation natural environment as the second test area, on this basis, the generalization of the trained electrical equipment change detection network can be made stronger.

[0065] The embodiment of the present application can reduce the impact of interference factors such as weather changes, seasonal cycle characteristics, and light brightness on the electrical equipment change detection network by using natural backgrounds in the same area at different times.

[0066] Step 302: Acquire multiple foreground image pairs.

[0067] Each foreground image pair includes a first foreground image and a second foreground image corresponding to the electrical device sample in different states. Different foreground image pairs may correspond to different electrical device samples or to the same electrical device sample. For different foreground image pairs corresponding to the same electrical device sample, the first foreground images of different foreground image pairs are different and / or the second foreground images are different.

[0068] As an example but not limitation, the state of the indicator light may include the indicator light being on and the indicator light being off. The first foreground image in a pair of foreground images of the indicator light may be an image representing that the indicator light is on, and the second foreground image may be an image representing that the indicator light is off. The first foreground image in another pair of foreground images of the indicator light may be an image representing that the indicator light is off, and the second foreground image may be an image representing that the indicator light is on.

[0069] In order to ensure the richness of the electrical equipment samples, the foreground image pair may be images of various states of different primary equipment, different secondary equipment, and different components.

[0070] Step 303: construct a training sample data set based on multiple background image pairs and multiple foreground image pairs.

[0071] The above-mentioned training sample data set may refer to a data set used to train an electrical equipment change detection network.

[0072] In a possible implementation, a training sample data set is constructed based on a plurality of background image pairs and a plurality of foreground image pairs, including:

[0073] For any electrical equipment sample, if the work of the electrical equipment sample needs to rely on other electrical equipment, then obtain images of the other electrical equipment, and synthesize the images of the other electrical equipment at the same position of the first background image and the second background image of any background image pair, to obtain the third background image and the fourth background image of the electrical equipment sample;

[0074] If the work of the electrical equipment sample does not depend on other electrical equipment, determining that the first background image and the second background image of any first background image pair are respectively the third background image and the fourth background image of the electrical equipment sample;

[0075] synthesizing the first foreground image and the second foreground image of the foreground image pair of the electrical device sample at the same position of the third background image and the fourth background image of the electrical device sample, respectively, to obtain a first synthesized image and a second synthesized image of the electrical device sample;

[0076] Determine that the first composite image is the first image sample of the corresponding electrical device sample, and the second composite image is the second image sample of the corresponding electrical device sample; or, determine that the first composite image is the first image sample of the corresponding electrical device sample, and the second background image is the second image sample of the corresponding electrical device sample; or, determine that the first background image is the first image sample of the corresponding electrical device sample, and the second composite image is the second image sample of the corresponding electrical device sample;

[0077] A training sample data set is constructed based on the first image samples and the second image samples of each electrical device sample.

[0078] As an example but not limitation, the electrical equipment sample is an indicator light. The indicator light in the substation depends on the switch cabinet, so the other electrical equipment may refer to the switch cabinet. The electrical equipment sample is a respirator. The respirator in the substation depends on the transformer, so the other electrical equipment may refer to the transformer.

[0079] Since there are many electrical equipment change scenarios in the substation, different electrical equipment change scenarios require different image synthesis methods. For example, by determining that the first background image is the first image sample of the corresponding electrical equipment sample, and the second synthetic image is the second image sample of the corresponding electrical equipment sample, a data set for training the newly added electrical equipment scene can be obtained. By determining that the first synthetic image is the first image sample of the corresponding electrical equipment sample, and the second background image is the second image sample of the corresponding electrical equipment sample, a data set for training the electrical equipment loss scenario can be obtained. By determining that the first synthetic image is the first image sample of the corresponding electrical equipment sample, and the second synthetic image is the second image sample of the corresponding electrical equipment sample, a data set for training the state change, color change, and electrical equipment damage of the electrical equipment can be obtained.

[0080] It should be noted that the above electrical equipment samples and other electrical equipment are all electrical equipment in the substation. The above other electrical equipment is the electrical equipment that the corresponding electrical equipment samples need to rely on when working.

[0081] For the convenience of description, the above-mentioned images of the electrical equipment samples and the images of other electrical equipment may be referred to as images of electrical equipment. The above-mentioned images of electrical equipment may be images extracted from the monitoring video of the substation. Since the images extracted from the monitoring video of the substation may include other contents in addition to the electrical equipment samples, the extracted images may be cut out to extract the images of the electrical equipment samples from the extracted images.

[0082] Since the cutout is usually random, that is, the sizes of two foreground images of the same electrical equipment sample may be different, the foreground image can be scaled to a suitable size to match the background image.

[0083] When the foreground image is synthesized into the background image, a random synthesis position can be obtained, and it is ensured that the foreground image position is within the range of the background image.

[0084] After the foreground image is synthesized with the background image, the outline of the foreground image in the synthesized image can be blurred to make the outline smoother and more natural. Then, the image harmonization algorithm can be used to solve the problem of color and lighting disharmony between the foreground image and the background image. By adjusting the color and lighting of the foreground image to make it more compatible with the background image, the synthesized image looks more harmonious.

[0085] In a possible implementation, constructing a training sample data set based on the first image sample and the second image sample of each electrical device sample includes:

[0086] Based on the first image sample and the second image sample of each electrical device sample, determining a change label of each electrical device sample, wherein the change label of each electrical device sample includes a true value of a heat map of a sample change region of each electrical device sample, a true value of a center point offset of the sample change region, and a true value of width and height information of the sample change region, including that the sample change region is a region where the second image sample of the corresponding electrical device sample changes relative to the first image sample;

[0087] A training sample data set is constructed based on the first image sample, the second image sample and the change label of each electrical device sample.

[0088] In one possible implementation, the true value of the heat map of the corresponding sample change area, the true value of the center point offset of the sample change area, and the true value of the width and height information can be determined according to the synthesis position of the foreground image and the size of the foreground image, thereby obtaining the change label of each electrical equipment sample.

[0089] The embodiment of the present application can generate training sample data sets in batches by randomly using different background image pairs, randomly adopting the above-mentioned different image synthesis methods and saving corresponding labels.

[0090] In order to increase the diversity of the training sample data set and reduce the dependence of the electrical equipment change detection network on certain attributes, and improve the generalization ability of the electrical equipment change detection network, data enhancement can be performed randomly on the first image sample, the second image sample and the change label.

[0091] The above data enhancement process can include:

[0092] The first image sample is randomly cropped slightly, and the second image sample and the change label are randomly cropped synchronously. The cropping direction of the second image sample and the change label is opposite to that of the first image sample to better simulate the jitter that occurs when the camera captures the image. As an example but not a limitation, the width and height information of the image is set to (w, h), the random range of the aspect ratio r after cropping is set to (a, 1), and the width and height information after cropping (w', h') = a*(w, h). The empirical value of a is 0.9, r of the first image sample is any random number between a and 1, and r of the second image sample and the change label needs to be consistent.

[0093] The above data enhancement process can also include:

[0094] The first image sample is not processed, and the second image sample and the change label are synchronously rotated, and the rotation angle can be a value within 5 degrees.

[0095] Since the above data enhancement is random, the training sample data set can be divided into a data set without data enhancement and a data set with data enhancement. These two data sets are input into the electrical equipment change detection network to obtain the predicted value of the sample change area.

[0096] Step 304: train the electrical equipment change detection network based on the training sample data set to obtain a trained electrical equipment change detection network.

[0097] The loss is summed using the heat map, center point offset, and width and height information prediction values ​​of the sample change area obtained by the electrical equipment change detection network, as well as the true values ​​of the heat map, center point offset, and width and height information obtained based on the change label. The loss function consists of three losses: bias loss (reg_loss), heat map loss (hm_loss), and width and height loss (wh_loss). Based on the loss function, the parameters of the electrical equipment change detection network training process can be adjusted. If the loss value no longer decreases during the training process, the training of the electrical equipment change detection network is completed.

[0098] loss=reg_loss+hm_loss+wh_loss

[0099] Among them, hm_loss is focal_loss, which means the dynamically scaled cross entropy loss. Through a dynamic scaling factor, the weight of easily distinguishable samples can be dynamically reduced during training, so as to quickly focus on those difficult-to-distinguish samples.

[0100] hm_loss = FL(p t )=-(1-p t ) γ log(p t)

[0101] Among them, γ is a parameter used to reduce the loss contribution of easy-to-separate samples. t is the probability that the model predicts that belongs to the foreground.

[0102] Both reg_loss and wh_loss are L1_loss.

[0103]

[0104] Among them, f(x i ) and y i They represent the predicted value and the corresponding true value of the i-th sample respectively, and n is the number of samples.

[0105] Step 103: if the electrical device change detection result indicates that an electrical device change has occurred, a detection frame for marking the target change area is generated in the second image to be tested based on the center point information and width and height information of the target change area in the electrical device change detection result.

[0106] The target change region is a region in the second image to be tested where the electrical device changes relative to the first image to be tested.

[0107] In order to reduce the occurrence of multiple detections and missed detections in the results of electrical equipment change detection, the detection boxes can be deleted and merged according to certain screening rules to reduce the false detection probability of the electrical equipment change detection network and improve the robustness of the change detection method.

[0108] In a possible implementation, the pixel area of ​​each detection frame is calculated;

[0109] If there is a detection frame whose pixel area is smaller than the preset area, the detection frame is deleted from the electrical device change detection result.

[0110] Optionally, the preset area may refer to a preset pixel area threshold, which may be set according to the size of the target to be detected. As an example but not a limitation, at a resolution of 1920*1080, the preset area may be 500 pixels squared.

[0111] In a possible implementation manner, when the number of detection frames is at least two, the method further includes:

[0112] Detect whether there are adjacent detection frames in at least two detection frames;

[0113] If there are adjacent detection boxes, merge them.

[0114] By merging adjacent detection frames, the electrical equipment change detection result can be fine-tuned, thereby making the final electrical equipment change detection result more accurate and intuitive.

[0115] In a possible implementation, detecting whether there are adjacent detection frames in at least two detection frames includes:

[0116] For a first detection frame and a second detection frame, the first detection frame is any detection frame of the at least two detection frames, and the second detection frame is any detection frame of the at least two detection frames except the first detection frame, and a minimum distance between the first detection frame and the second detection frame is calculated;

[0117] Determine a distance threshold between the first detection frame and the second detection frame based on the width and height information of the first detection frame and the width and height information of the second detection frame;

[0118] If the minimum distance is less than the distance threshold, determining that the first detection frame and the second detection frame are adjacent detection frames;

[0119] If the minimum distance is greater than or equal to the distance threshold, it is determined that the first detection frame and the second detection frame are not adjacent detection frames.

[0120] In a possible implementation, determining a distance threshold between the first detection frame and the second detection frame based on the width and height information of the first detection frame and the width and height information of the second detection frame includes:

[0121] By formula Calculate the distance threshold;

[0122] Wherein, l represents the distance threshold, w1 represents the width of the first detection frame, h1 represents the height of the first detection frame, w2 represents the width of the second detection frame, h2 represents the height of the second detection frame, and λ represents the adjustment coefficient.

[0123] λ is an adjustable parameter that can be preset. Optionally, it can be set to 1.2 based on empirical values.

[0124] When the electrical equipment change detection network in this embodiment detects a change in the electrical equipment, it outputs the center point information and the width and height information of the changed area. A detection frame can be generated based on the center point information and the width and height information of the changed area. Based on the detection frame, the changed area can be marked in the second image to be tested without the need for fine pixel-level segmentation of the second image to be tested, thereby reducing the complexity of change detection of electrical equipment and improving the efficiency of change detection of electrical equipment.

[0125] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0126] Corresponding to the electrical equipment change detection method described in the above embodiment, Figure 4A schematic diagram of the structure of an electrical equipment change detection device provided in an embodiment of the present application is shown. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown.

[0127] Reference Figure 4 , the device comprises:

[0128] An image acquisition module 41 is used to acquire a pair of images to be tested of the same first area to be tested in the substation at different times, wherein the pair of images to be tested includes a first image to be tested and a second image to be tested, and the shooting time of the second image to be tested is later than the shooting time of the first image to be tested;

[0129] A change detection module 42, configured to input the image pair to be tested into an electrical device change detection network, and obtain an electrical device change detection result of the second image to be tested relative to the first image to be tested;

[0130] The detection frame generation module 43 is used to generate a detection frame for marking the target change area in the second image to be tested if the electrical equipment change detection result indicates that the electrical equipment change has occurred, based on the center point information and width and height information of the target change area in the electrical equipment change detection result. The target change area is the area in the second image to be tested where the electrical equipment change has occurred relative to the first image to be tested.

[0131] Optionally, the electrical equipment change detection network includes a target processing layer, a calculation module, a target convolution layer and a prediction head of a CenterNet target detection network in sequence, and the target processing layer includes a CNN backbone network and a bi-temporal image converter of a BIT_CD change detection network in sequence; the change detection module 42 includes:

[0132] A first input unit, used for inputting the image pair to be tested into the target processing layer to obtain a first feature map of the first image to be tested and a second feature map of the second image to be tested;

[0133] a triggering unit, configured to trigger the calculation module to calculate an absolute value of a difference between the first characteristic map and the second characteristic map to obtain a third characteristic map;

[0134] A second input unit is used to input the third feature map into the target convolution layer to obtain a fourth feature map, wherein the size and the number of channels of the fourth feature map match those of the prediction head;

[0135] The third input unit is used to input the fourth feature map into the prediction head to perform electrical device change detection to obtain the electrical device change detection result.

[0136] Optionally, the prediction head includes a first convolution layer, a second convolution layer, a third convolution layer and a generation module; the third input unit is specifically used for:

[0137] Input the fourth feature map into the first convolutional layer to obtain a heat map of electrical device changes, and when the heat map indicates that electrical device changes have occurred, determine from the heat map the position information of the center point of the target change area in the fourth feature map;

[0138] Inputting the fourth feature map into the second convolutional layer to obtain the center point offset of the target change area;

[0139] Inputting the fourth feature map into the third convolutional layer to obtain width and height information of the target change area;

[0140] The generation module is triggered to generate the electrical device change detection result based on the position information of the center point of the target change area in the fourth feature map, the offset of the center point of the target change area, and the width and height information of the target change area.

[0141] Optionally, the above device further includes:

[0142] A first acquisition module is used to acquire a plurality of background image pairs, each background image pair includes a first background image and a second background image of the same second area to be measured at different times, and the shooting time of the second background image is later than the shooting time of the corresponding first background image;

[0143] A second acquisition module is used to acquire a plurality of foreground image pairs, each foreground image pair including a first foreground image and a second foreground image corresponding to the electrical device sample in different states;

[0144] A sample construction module, used to construct a training sample data set based on the multiple background image pairs and the multiple foreground image pairs;

[0145] The network training module is used to train the electrical equipment change detection network based on the training sample data set to obtain the trained electrical equipment change detection network.

[0146] Optionally, the sample building blocks include:

[0147] An image acquisition unit is used for acquiring images of any of the electrical equipment samples if the operation of the electrical equipment sample depends on other equipment samples, and synthesizing the images of the other equipment samples at the same position of the first background image and the second background image of any of the background image pairs to obtain a third background image and a fourth background image of the electrical equipment sample;

[0148] a first determining unit, configured to determine that the first background image and the second background image of any first background image pair are respectively the third background image and the fourth background image of the electrical device sample if the electrical device sample does not need to rely on the other device samples for operation;

[0149] an image synthesis unit, configured to synthesize the first foreground image and the second foreground image of the foreground image pair of the electrical device sample at the same position of the third background image and the fourth background image of the electrical device sample, respectively, to obtain a first synthesized image and a second synthesized image of the electrical device sample;

[0150] A second determining unit is used to determine that the first synthetic image is a first image sample of the corresponding electrical device sample, and the second synthetic image is a second image sample of the corresponding electrical device sample; or, determine that the first synthetic image is a first image sample of the corresponding electrical device sample, and the second background image is a second image sample of the corresponding electrical device sample; or, determine that the first background image is a first image sample of the corresponding electrical device sample, and the second synthetic image is a second image sample of the corresponding electrical device sample;

[0151] The sample construction unit is used to construct the training sample data set based on the first image sample and the second image sample of each electrical device sample.

[0152] Optionally, the sample construction unit is specifically used for:

[0153] Based on the first image sample and the second image sample of each electrical device sample, determining a change label of each electrical device sample, wherein the change label of each electrical device sample includes a true value of a heat map of a sample change region of each electrical device sample, a true value of a center point offset of the sample change region, and a true value of width and height information of the sample change region, including that the sample change region is a region where the second image sample of the corresponding electrical device sample changes relative to the first image sample;

[0154] The training sample data set is constructed based on the first image sample, the second image sample and the change label of each electrical device sample.

[0155] Optionally, the above device further comprises:

[0156] An area calculation module is used to calculate the pixel area of ​​each detection frame;

[0157] The detection frame deletion module is used to delete the detection frame from the change detection result if there is a detection frame whose pixel area is smaller than a preset area.

[0158] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0159] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device 5 of this embodiment includes: at least one processor 50 ( Figure 5 Only one is shown in the figure) a processor, a memory 51, and a computer program 52 stored in the memory 51 and executable on the at least one processor 50, and when the processor 50 executes the computer program 52, the steps in any of the above-mentioned electrical device change detection method embodiments are implemented.

[0160] The electronic device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will appreciate that Figure 5 It is only an example of the electronic device 5 and does not constitute a limitation on the electronic device 5. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0161] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0162] In some embodiments, the memory 51 may be an internal storage unit of the electronic device 5, such as a hard disk or memory of the electronic device 5. In other embodiments, the memory 51 may also be an external storage device of the electronic device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 5. Further, the memory 51 may also include both an internal storage unit and an external storage device of the electronic device 5. The memory 51 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0163] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0164] An embodiment of the present application also provides an electronic device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor implements the steps of any of the above-mentioned method embodiments when executing the computer program.

[0165] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0166] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0167] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0168] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0169] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0170] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0171] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0172] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting changes in electrical equipment, characterized in that: include: Acquire a pair of images to be tested of the same first area to be tested in the substation at different times, the pair of images to be tested comprising a first image to be tested and a second image to be tested, wherein the shooting time of the second image to be tested is later than the shooting time of the first image to be tested; Inputting the image pair to be tested into an electrical device change detection network to obtain an electrical device change detection result of the second image to be tested relative to the first image to be tested; If the electrical equipment change detection result indicates that an electrical equipment change has occurred, a detection frame for marking the target change area is generated in the second image to be tested based on the center point information and width and height information of the target change area in the electrical equipment change detection result, and the target change area is the area in the second image to be tested where the electrical equipment change has occurred relative to the first image to be tested.

2. The method according to claim 1, characterized in that The electrical equipment change detection network sequentially includes a target processing layer, a computing module, a target convolution layer, and a prediction head of a CenterNet target detection network, and the target processing layer sequentially includes a CNN backbone network and a bi-temporal image converter of a BIT_CD change detection network; The step of inputting the image pair to be tested into an electrical device change detection network to obtain an electrical device change detection result comprises: Inputting the image pair to be tested into the target processing layer to obtain a first feature map of the first image to be tested and a second feature map of the second image to be tested; Triggering the calculation module to calculate the absolute value of the difference between the first characteristic map and the second characteristic map to obtain a third characteristic map; Inputting the third feature map into the target convolution layer to obtain a fourth feature map, wherein the size and the number of channels of the fourth feature map match those of the prediction head; The fourth feature map is input into the prediction head to perform electrical device change detection to obtain the electrical device change detection result.

3. The method according to claim 2, characterized in that The prediction head includes a first convolutional layer, a second convolutional layer, a third convolutional layer and a generation module; Inputting the fourth feature map into the prediction head to perform electrical device change detection, and obtaining the electrical device change detection result, comprises: Input the fourth feature map into the first convolutional layer to obtain a heat map of electrical device changes, and when the heat map indicates that electrical device changes have occurred, determine from the heat map the position information of the center point of the target change area in the fourth feature map; Inputting the fourth feature map into the second convolutional layer to obtain the center point offset of the target change area; Inputting the fourth feature map into the third convolutional layer to obtain width and height information of the target change area; The generation module is triggered to generate the electrical device change detection result based on the position information of the center point of the target change area in the fourth feature map, the offset of the center point of the target change area, and the width and height information of the target change area.

4. The method according to claim 3, characterized in that: The training process of the electrical equipment change detection network includes: Acquire multiple background image pairs, each background image pair comprising a first background image and a second background image of the same second area to be measured at different times, wherein the shooting time of the second background image is later than the shooting time of the corresponding first background image; Acquire a plurality of foreground image pairs, each foreground image pair comprising a first foreground image and a second foreground image corresponding to the electrical device sample in different states; Constructing a training sample data set based on the multiple background image pairs and the multiple foreground image pairs; Based on the training sample data set, the electrical equipment change detection network is trained to obtain the trained electrical equipment change detection network.

5. The method according to claim 4, characterized in that The step of constructing a training sample data set based on the multiple background image pairs and the multiple foreground image pairs includes: For any of the electrical equipment samples, if the operation of the electrical equipment sample needs to rely on other equipment samples, images of the other equipment samples are obtained, and the images of the other equipment samples are respectively synthesized at the same position of the first background image and the second background image of any of the background image pairs to obtain a third background image and a fourth background image of the electrical equipment sample; If the operation of the electrical device sample does not depend on the other device samples, determining that the first background image and the second background image of any one of the first background image pairs are the third background image and the fourth background image of the electrical device sample respectively; respectively synthesizing a first foreground image and a second foreground image of the foreground image pair of the electrical device sample at the same position of a third background image and a fourth background image of the electrical device sample to obtain a first synthesized image and a second synthesized image of the electrical device sample; Determine that the first synthetic image is the first image sample of the corresponding electrical device sample, and the second synthetic image is the second image sample of the corresponding electrical device sample; or, determine that the first synthetic image is the first image sample of the corresponding electrical device sample, and the second background image is the second image sample of the corresponding electrical device sample; or, determine that the first background image is the first image sample of the corresponding electrical device sample, and the second synthetic image is the second image sample of the corresponding electrical device sample; The training sample data set is constructed based on the first image samples and the second image samples of each electrical device sample.

6. The method according to claim 5, characterized in that The step of constructing the training sample data set based on the first image sample and the second image sample of each electrical device sample includes: Based on the first image sample and the second image sample of each electrical device sample, determining a change label of each electrical device sample, wherein the change label of each electrical device sample includes a true value of a heat map of a sample change region of each electrical device sample, a true value of a center point offset of the sample change region, and a true value of width and height information of the sample change region, including that the sample change region is a region where the second image sample of the corresponding electrical device sample changes relative to the first image sample; The training sample data set is constructed based on the first image sample, the second image sample and the change label of each electrical device sample.

7. The method according to any one of claims 1 to 6, characterized in that: Also includes: Calculate the pixel area of ​​each detection box; If there is a detection frame whose pixel area is smaller than the preset area, the detection frame is deleted from the electrical device change detection result.

8. A change detection device for electrical equipment, characterized in that: include: An image acquisition module, used to acquire a pair of images to be tested of the same first area to be tested in the substation at different times, wherein the pair of images to be tested includes a first image to be tested and a second image to be tested, and the shooting time of the second image to be tested is later than the shooting time of the first image to be tested; a change detection module, configured to input the image pair to be tested into an electrical device change detection network, and obtain an electrical device change detection result of the second image to be tested relative to the first image to be tested; A detection frame generation module is used to generate a detection frame for marking the target change area in the second image to be tested if the electrical equipment change detection result indicates that the electrical equipment change has occurred, based on the center point information and width and height information of the target change area in the electrical equipment change detection result, wherein the target change area is the area in the second image to be tested where the electrical equipment change has occurred relative to the first image to be tested.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images