A method and system for generating missing bolt pins of power transmission lines

The noise probability diffusion model is used to iteratively generate the missing images of the transmission line bolt pin needle in the Gaussian distribution, which solves the problem of difficulty in generating samples in the prior art and improves the performance of deep learning technology in the detection of missing pin needle of bolt pin.

CN116363090BActive Publication Date: 2024-11-01STATE GRID SHANDONG ELECTRIC POWER CO +1
View PDF 12 Cites 0 Cited by

Patent Information

Application Number
CN202310294234.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-03-21
Publication Date
2024-11-01
Estimated Expiration
2043-03-21

AI Technical Summary

Technical Problem

The prior art is difficult to effectively generate sample images of the bolt pin needle missing from the transmission line, which results in the training of the bolt pin missing defect identification model requires a lot of manpower and material resources, and generative adversarial networks are difficult to apply in this field.

Method used

A noise probability diffusion model is introduced to generate missing images of the power line bolt pin needle through image completion. The forward noise addition and reverse noise denoising processes are used to iterate the image in the Gaussian distribution to improve the quality of the generated results.

Benefits of technology

High-quality bolt pin missing image generation is achieved, simplifying the model training process and improving the performance of deep learning technology in defect detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116363090B_ABST
    Figure CN116363090B_ABST
Patent Text Reader

Abstract

The present invention belongs to the technical field of intelligent analysis of missing images, and specifically relates to a method and system for generating missing pins of bolts of a transmission line, comprising: obtaining an original image of a transmission line bolt and a positioning mask of a bolt pin; performing image and operation on the obtained original image and the positioning mask to obtain an image to be processed; performing missing generation of the image to be processed according to the obtained image to be processed and a preset missing generation model, and completing the generation of missing pins of bolts of a transmission line; wherein, in the process of missing generation, the preset missing generation model sequentially performs forward denoising and reverse denoising on the image to be processed to complete the missing generation of the image to be processed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of missing image intelligent analysis, and in particular relates to a method and system for generating missing bolt pins for a power transmission line. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] In the power system, transmission lines occupy an extremely important position, and the safety of transmission lines is the top priority of the entire power safety field. In the transmission line, bolts play the role of reinforcing various components of the transmission line, so it is very important to ensure the integrity and safety of the bolts. A complete bolt is mainly composed of a nut, a nut and a pin. Only when the three are at the specified position of the transmission line at the same time can the safety of the transmission system be better guaranteed. In actual operation scenarios, the pins of bolts often fall off under long-term working conditions, which undoubtedly brings great hidden dangers to the transmission line. Therefore, it is necessary to detect the missing defects of bolt pins in the transmission line in time. However, in order to train the bolt pin missing defect recognition model, it is often necessary to collect a sufficient number of bolt pin missing sample images in advance, which undoubtedly requires huge manpower and material resources. Therefore, how to design an algorithm to automatically generate transmission line bolt pin missing image samples becomes particularly important.

[0004] At present, image defect generation has attracted more and more attention, and the main solution is to use generative adversarial networks (GAN). Generative adversarial networks are a deep learning model that mainly includes a generative model and a discriminative model, that is, the generator and discriminator of the corresponding neural network. In this framework, data is generated by the generator G; the discriminator is used to determine whether the image is real or machine-generated, in order to determine whether the data is "fake data" made by the generator; the generator and the discriminator are used to confront each other and continuously adjust the parameters; so that the discriminative network cannot determine whether the output of the generated network is true, so as to obtain a more accurate defect generator. In practical applications, generative adversarial networks often need to rely on a large amount of data and are difficult to train and converge, so it is difficult to apply them to the field of generating samples of missing bolt pins on transmission lines. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a method and system for generating missing bolt pins for transmission lines, introduces a noise probability diffusion model into the generation of missing bolt pins, and improves the performance of defect detection using deep learning technology.

[0006] According to some embodiments, a first solution of the present invention provides a method for generating missing bolt pins for a transmission line, using the following technical solution:

[0007] A method for generating missing bolt pins of a power transmission line, comprising:

[0008] Obtaining the original image of the transmission line bolts and the bolt pin positioning mask;

[0009] Performing image and operation on the acquired original image and the positioning mask to obtain an image to be processed;

[0010] According to the obtained image to be processed and the preset defect generation model, the defect generation of the image to be processed is performed to complete the generation of the bolt pin defect of the transmission line;

[0011] Among them, in the process of defect generation, the preset defect generation model performs forward denoising and reverse denoising on the image to be processed in sequence to complete the defect generation of the image to be processed.

[0012] As a further technical limitation, the original image of the transmission line bolts obtained contains bolt pins.

[0013] As a further technical limitation, in the process of obtaining the bolt pin positioning mask image, a black image mask with the same size as the original image is obtained, and the pin position in the black image mask is roughly filled with white to complete the manual annotation of the image and obtain the bolt pin positioning mask image.

[0014] As a further technical limitation, the positioning mask and the original image are used to perform image operations, the image information of the positions where the mask pixels are black in the original image is retained, and the original image information of the positions of the white areas is removed, so as to obtain the image to be processed.

[0015] As a further technical limitation, the defect generation model adopts a noise diffusion probability model.

[0016] Furthermore, the image is converted into a Gaussian distribution by adding noise to the image to be processed, and is re-projected into the image space by using a denoising process, thereby realizing the image generation process.

[0017] As a further technical qualification, the noise image x is sampled from a Gaussian distribution t By merging the channels of the image to be processed and the noise as the network input, the noise image x can be obtained. t-1 .After T times of loop iteration, the final noise-free image x0 can be obtained.

[0018] According to some embodiments, a second solution of the present invention provides a transmission line bolt pin missing generation system, which adopts the following technical solution:

[0019] A transmission line bolt pin missing generation system, comprising:

[0020] An acquisition module, which is configured to acquire an original image of a transmission line bolt and a bolt pin positioning mask;

[0021] A calculation module, which is configured to perform an image and operation on the acquired original image and the positioning mask to obtain an image to be processed;

[0022] A generation module is configured to perform defect generation of the image to be processed according to the obtained image to be processed and a preset defect generation model, and complete the generation of the bolt pin defect of the transmission line;

[0023] Among them, in the process of defect generation, the preset defect generation model performs forward denoising and reverse denoising on the image to be processed in sequence to complete the defect generation of the image to be processed.

[0024] According to some embodiments, a third solution of the present invention provides a computer-readable storage medium, which adopts the following technical solution:

[0025] A computer-readable storage medium stores a program, which, when executed by a processor, implements the steps in the method for generating missing bolt pins for a power transmission line as described in the first embodiment of the present invention.

[0026] According to some embodiments, a fourth solution of the present invention provides an electronic device, which adopts the following technical solution:

[0027] An electronic device comprises a memory, a processor and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for generating missing bolt pins of a transmission line as described in the first embodiment of the present invention are implemented.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] The present invention cleverly uses the image completion character to complete the generation of bolt defects in the transmission line; uses the latest noise diffusion probability model, utilizes the noise probability diffusion model to generate independent Gaussian distribution by forward noise addition and reverse denoising to generate images to achieve the task of generating transmission line bolt pin defects, and utilizes the iterative process to well ensure the image quality of the generated results. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The drawings in the specification that constitute a part of this embodiment are used to provide a further understanding of this embodiment. The schematic embodiments of this embodiment and their descriptions are used to explain this embodiment and do not constitute improper limitations on this embodiment.

[0031] Figure 1 It is a flow chart of the method for generating missing bolt pins of a transmission line in the first embodiment of the present invention;

[0032] Figure 2 The original image of the transmission line bolt in the first embodiment of the present invention;

[0033] Figure 3 is the image to be processed in the first embodiment of the present invention;

[0034] Figure 4 It is a schematic diagram of the result of generating the missing bolt pin of the transmission line in the first embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0036] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0037] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0038] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0039] Embodiment 1

[0040] Embodiment 1 of the present invention introduces a method for generating missing bolt pins of a power transmission line.

[0041] like Figure 1 A method for generating missing bolt pins of a power transmission line is shown, comprising:

[0042] Manually annotate the transmission line bolt images to obtain positioning masks;

[0043] Use the positioning mask to create a training data set;

[0044] Build a noise probability diffusion model and conduct network model training;

[0045] The trained network model is used to generate actual transmission line bolt defects.

[0046] As one or more implementation modes, the original image of the transmission line bolts obtained is as follows: Figure 2 As shown, a black image mask of the same size as the image with the pins in the original image of the transmission line bolt is established, and the pin positions are roughly filled with white to obtain a positioning mask for the pin positions.

[0047] As one or more implementation methods, the positioning mask and the original image are used to perform image and operation, retain the image information of the black position of the mask pixel in the original image, and remove the original image information of the white area position, so as to obtain Figure 3 The images to be processed shown are constructed by establishing a large number of pairs of images to be processed and original images to form a training data set.

[0048] As one or more implementation methods, the specific steps of building a noise probability diffusion model and performing network model training are as follows:

[0049] Using the acquired training data set, a noise probability diffusion model is constructed and model training is performed. The model converts the image into a Gaussian distribution by adding noise to the original image, and reprojects it into the image space by using the denoising process, thereby realizing the image generation process; its basic process is as follows:

[0050] Step 1: Construct the forward noise adding process of the noise probability diffusion model. Given a data distribution x0~q(x0), a forward noise adding process q can be defined, with a variance of β added over time t t Gaussian noise to generate noise image x1 to noise image x T , its specific form is as follows:

[0051]

[0052] Step 2: Construct the noise probability diffusion model to reverse the denoising process. Given a sufficiently large time T and variance β t In the case of a noisy image x t can be regarded as an independent Gaussian distribution. Therefore, assuming that we know the inverse distribution q(x t-1 |x t ) can be sampled from a Gaussian distribution t , perform the inverse process to obtain the sampling of the noise-free image q(x0). Then since q(x t-1 |x t ) depends on the entire data distribution and is difficult to obtain directly. Therefore, a neural network p is used to estimate the process. The specific form is as follows:

[0053]

[0054] The combination of q and p can be regarded as a variational autoencoder, so its variational lower bound can be expressed as follows:

[0055] L vlb :=L0+L1+...+L T-1 +L T

[0056] L0: =-logp θ (x0|x1)

[0057] L t-1 :=D KL (q(x t-1 |x t ,x0)||p θ (x t-1 |x t ))

[0058] L T :=D KL (q(x T |x0)||p(x T ))

[0059] Using this variational lower bound, we can view it as a loss function of p, and use the cross entropy between q and p at different stages to complete the training of p. Using this form, we can use the original image as x0 and the image to be processed as the conditional prior of the noise probability diffusion model to complete the training of the network model.

[0060] It should be noted that the noise probability diffusion model used in this embodiment is actually a mathematical operation of forward noise addition and reverse denoising. The image is projected into a noise on a Gaussian distribution by adding noise, and then the noise is restored to an image by denoising. The process of image generation is the process of converting noise into an image.

[0061] As one or more implementation methods, the specific steps of using the trained network model to generate actual transmission line bolt defects are as follows:

[0062] Using the obtained neural network model p, we can get the noise image x t Get the noise-free image x0. First, sample the noise image x from the Gaussian distribution t Then, the image to be processed is used as the conditional prior of the noise probability diffusion model and introduced into the network reasoning to obtain the noise image x t-1 After T iterations, the final noise-free image x0 can be obtained. The final result is as follows: Figure 4 shown.

[0063] The present embodiment provides a method for generating missing bolt pins for transmission lines based on a noise probability diffusion model, and uses image completion to generate samples of missing bolt pins in transmission lines, so as to improve the performance of subsequent detection of the defect using deep learning technology. First, the positions of the bolt pins in the original image are manually marked to obtain their positioning masks, and the mask is used to deduct the input image that needs to be retained from the original image, and a large number of image pairs are obtained. A training data set is constructed using the image to be input and the original image, which is used to train the noise diffusion probability model. After the model is fully trained, the image to be input is used as the model input to generate a missing bolt pin defect. The present invention is simple and easy to implement, and has significant effects. It can effectively generate missing bolt pin defects to better assist the target detection algorithm based on deep learning for the defect.

[0064] Embodiment 2

[0065] Embodiment 2 of the present invention introduces a system for generating missing bolt pins for a power transmission line.

[0066] A transmission line bolt pin missing generation system, comprising:

[0067] An acquisition module, which is configured to acquire an original image of a transmission line bolt and a bolt pin positioning mask image;

[0068] A computing module, which is configured to perform image computing on the acquired original image and the positioning mask image to obtain an image to be processed;

[0069] A generation module is configured to perform defect generation of the image to be processed according to the obtained image to be processed and a preset defect generation model, and complete the generation of the bolt pin defect of the transmission line;

[0070] Among them, in the process of defect generation, the preset defect generation model adopts the noise diffusion probability model, and uses the noise probability diffusion model to perform forward noise addition and reverse denoising on the image to be processed in turn to complete the defect generation of the image to be processed.

[0071] The detailed steps are the same as the method for generating missing bolt pins for transmission lines provided in Example 1, and will not be repeated here.

[0072] Embodiment 3

[0073] Embodiment 3 of the present invention provides a computer-readable storage medium.

[0074] A computer-readable storage medium stores a program, which, when executed by a processor, implements the steps in the method for generating missing bolt pins for a transmission line as described in Embodiment 1 of the present invention.

[0075] The detailed steps are the same as the method for generating missing bolt pins for transmission lines provided in Example 1, and will not be repeated here.

[0076] Embodiment 4

[0077] A fourth embodiment of the present invention provides an electronic device.

[0078] An electronic device comprises a memory, a processor and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for generating missing bolt pins for a transmission line as described in the first embodiment of the present invention are implemented.

[0079] The detailed steps are the same as the method for generating missing bolt pins for transmission lines provided in Example 1, and will not be repeated here.

[0080] The above description is only a preferred embodiment of the present embodiment and is not intended to limit the present embodiment. For those skilled in the art, the present embodiment may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present embodiment shall be included in the protection scope of the present embodiment.

Claims

1. A method for generating missing bolt pins of a power transmission line, characterized in that: include: Obtaining the original image of the transmission line bolts and the bolt pin positioning mask; Performing image and operation on the acquired original image and the positioning mask to obtain an image to be processed; According to the obtained image to be processed and the preset defect generation model, the defect generation of the image to be processed is performed to complete the generation of the bolt pin defect of the transmission line; In the process of defect generation, the preset defect generation model sequentially performs forward denoising and reverse denoising on the image to be processed to complete the defect generation of the image to be processed; The missing generation model is specifically: building a noise probability diffusion model, performing network model training, and using the acquired training data set to train the model. The model converts the image into a Gaussian distribution by adding noise to the original image, and re-projects it into the image space by using the denoising process to realize the image generation process; The basic process of the noise probability diffusion model is as follows: construct the noise probability diffusion model forward noise addition process: given the data distribution x0~q(x0), define the forward noise addition process q, add the variance β as time t t Gaussian noise to generate noise image x1 to noise image x T , its specific form is as follows: Construct the inverse denoising process of the noise probability diffusion model: Given a sufficiently large time T and variance β t In the case of a noisy image x t Approximately independent Gaussian distribution; known inverse distribution q(x t-1 |x t ), sample x from a Gaussian distribution t , perform the inverse process to obtain the sampling of the noise-free image q(x0); use the neural network p to estimate q(x t-1 |x t ) The process is specifically as follows: The combination of q and p is a variational autoencoder with a variational lower bound of: L vlb :=L0+L1+...+L T-1 +L T L0:=-logp θ (x0|x1) L t-1 :=D KL (q(x t-1 |x t ,x0)||p θ (x) t-1 |x t )) L T :=D KL (q(x T |x0)||p(x T )) By using the cross entropy between q and p at different stages, the training of p is completed; the image to be processed is used as the conditional prior of the noise probability diffusion model to complete the training of the network model.

2. A method for generating missing bolt pins for a power transmission line as claimed in claim 1, characterized in that: The original image of the transmission line bolts obtained contains bolt pins.

3. A method for generating missing bolt pins for a power transmission line as claimed in claim 1, characterized in that: In the process of obtaining the bolt pin positioning mask, a black image mask having the same size as the original image is obtained, and the pin position in the black image mask is roughly filled with white to complete the manual labeling of the image and obtain the bolt pin positioning mask.

4. A method for generating missing bolt pins for a power transmission line as claimed in claim 1, characterized in that: The positioning mask and the original image are used to perform image operations, the image information of the positions where the mask pixels are black in the original image is retained, and the original image information of the positions of the white areas is removed, so as to obtain the image to be processed.

5. A method for generating missing bolt pins for a power transmission line as claimed in claim 1, characterized in that: The missing generation model adopts a noise diffusion probability model.

6. A method for generating missing bolt pins for a power transmission line as claimed in claim 5, characterized in that: The image generation process is achieved by converting the image into a Gaussian distribution through the process of adding noise to the image to be processed, and then re-projecting it into the image space through the process of denoising.

7. A method for generating missing bolt pins for a power transmission line as claimed in claim 1, characterized in that: Sample a noisy image x from a Gaussian distribution t , by merging the channels of the image to be processed and the noise as the network input, we get the noisy image x t-1 .Through T loop iterations, the final noise-free image x0 is obtained.

8. A transmission line bolt pin missing generation system, characterized in that: include: An acquisition module, which is configured to acquire an original image of a transmission line bolt and a bolt pin positioning mask; A calculation module, which is configured to perform an image and operation on the acquired original image and the positioning mask to obtain an image to be processed; A generation module is configured to perform defect generation of the image to be processed according to the obtained image to be processed and a preset defect generation model, and complete the generation of the bolt pin defect of the transmission line; In the process of defect generation, the preset defect generation model sequentially performs forward denoising and reverse denoising on the image to be processed to complete the defect generation of the image to be processed; The missing generation model is specifically: building a noise probability diffusion model, performing network model training, and using the acquired training data set to train the model. The model converts the image into a Gaussian distribution by adding noise to the original image, and re-projects it into the image space by using the denoising process to realize the image generation process; The basic process of the noise probability diffusion model is as follows: construct the noise probability diffusion model forward noise addition process: given the data distribution x0~q(x0), define the forward noise addition process q, add the variance β as time t t Gaussian noise to generate noise image x1 to noise image x T , its specific form is as follows: Construct the inverse denoising process of the noise probability diffusion model: Given a sufficiently large time T and variance βt, the noise image x t Approximately independent Gaussian distribution; known inverse distribution q(x t-1 |x t ), sample x from a Gaussian distribution t , perform the inverse process to obtain the sampling of the noise-free image q(x0); use the neural network p to estimate q(x t-1 |x t ) The process is specifically as follows: The combination of q and p is a variational autoencoder with a variational lower bound of: L vlb :=L0+L1+...+L T-1 +L T L0:=-log p θ (x0||x1) L t-1 :=D KL (q(x t-1 |x t ,x0)||p θ (x) t-1 |x t )) L T :=D KL (q(x T |x0)||p(x T )) By using the cross entropy between q and p at different stages, the training of p is completed; the image to be processed is used as the conditional prior of the noise probability diffusion model to complete the training of the network model.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method for generating missing bolt pins for a transmission line as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps of the method for generating missing bolt pins for a transmission line as described in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Image restoration method and system based on edge restoration and content restoration

    CN110675339A

  • Self-cleaning device and method for power transmission line image monitoring system lens

    CN110996055A

  • Power transmission line bolt detection method and device, storage medium and electronic equipment

    CN112581445A

  • Image restoration method and device and electronic equipment

    CN112614066A

  • Small image multi-target detection method based on super-resolution

    CN113920013A