Method for generating ice-covered picture samples of transmission lines based on diffusion algorithm

Abstract

The present application relates to the field of artificial intelligence technology, and provides a method for generating transmission line icing picture samples based on a diffusion algorithm. The method includes the steps of: collecting initial sample pictures of transmission lines and first environmental information; identifying and extracting target elements and key elements in the initial sample pictures, and respectively annotating each of the target elements and the key elements based on the first environmental information to obtain first picture samples; writing each of the first picture samples into a model material library and triggering a picture-to-picture process to obtain new sample pictures; respectively annotating each of the new sample pictures and the initial sample pictures based on the first environmental information to obtain second picture samples. Based on this, picture-to-picture can be realized, the model training samples can be expanded, and the manual collection cost and risk can be reduced.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for generating transmission line icing picture samples based on a diffusion algorithm, a transmission line icing monitoring method, and a transmission line monitoring system. Background Art

[0002] With the rapid development of image processing and machine learning technologies, automated transmission line monitoring technologies have become an important part of the power industry.

[0003] Currently, transmission line icing detection mainly uses manual inspection combined with image recognition technology. The recognition model is trained with pictures collected manually. However, this method not only has a limited sample size, and the limitations in sample collection also lead to insufficient generalization ability of the trained model, but also there are safety hazards in manually collecting samples under extreme weather conditions. Summary of the Invention

[0004] In order to achieve automatic batch generation of sample pictures and reduce the manual sample collection burden, an embodiment of this application provides a method for generating transmission line icing picture samples based on a diffusion algorithm. The method includes the steps of: collecting initial sample pictures of transmission lines and first environmental information; identifying and extracting target elements and key elements in the initial sample pictures, and respectively annotating each of the target elements and the key elements based on the first environmental information to obtain first picture samples; wherein, the picture generation process includes generating the new sample pictures based on an image generation model, and the image generation model is implemented based on a diffusion algorithm; writing each of the first picture samples into a model material library and triggering the picture generation process to obtain new sample pictures; respectively annotating each of the new sample pictures and the initial sample pictures based on the first environmental information to obtain second picture samples.

[0005] Based on the above technical solution, multiple new sample pictures can be generated based on the initial sample pictures, realizing automatic expansion of model training samples. Moreover, in the picture generation process, the element basis used comes from real collected pictures, thus ensuring that the newly generated pictures are more in line with the real situation. Compared with directly relying on a large model for picture generation, the newly generated pictures can better meet the model training requirements.

[0006] In one embodiment, the picture generation process includes: retrieving historical first picture samples from the model material library whose annotation information matches the first environmental information; constructing a picture output requirement based on the target elements, the key elements, and the historical first picture samples; processing the output requirement based on an image generation model to obtain the new sample pictures.

[0007] Based on the above technical solution, obtaining a matching historical first picture sample from the model material library ensures that the elements used in the picture generation process are all from real pictures, and constructing the picture output requirements based on these real materials limits the picture generation range of the image generation model, thereby further ensuring that the output result is closer to the real situation.

[0008] In one implementation, the first environmental information includes temperature data and humidity data; the matching between the annotation information and the first environmental information includes: the differences between the temperature data and humidity data in the annotation information and the corresponding data in the first environmental information are all within the corresponding preset ranges.

[0009] Based on the above technical solution, by screening the historical first picture samples according to the temperature and humidity data, historical elements with the same or similar environmental information as the first picture sample are obtained, thereby ensuring the rationality and relevance of each element in the new sample picture and improving the sample quality.

[0010] In one implementation, the temperature data and humidity data are collected by sensors installed on the unmanned aerial vehicle when shooting the initial sample picture.

[0011] Based on the above technical solution, it can be ensured that the collected temperature and humidity data are consistent with the relevant data in the environment where the transmission line is located, which is more real and reliable than obtaining these data according to meteorological information.

[0012] In one implementation, the first environmental information includes altitude data, and the method further includes: screening the new sample picture based on the altitude data.

[0013] Based on the above technical solution, screening the new sample picture according to the altitude data can improve the rationality of the new sample picture used to generate training samples.

[0014] Based on the same inventive concept, an embodiment of the present application further provides a transmission line icing monitoring method, which is applied to a transmission line monitoring system. The system includes a data acquisition device and a data analysis device. The method includes: the data analysis device receives the real-time image and the second environmental information of the transmission line to be monitored sent by the data acquisition device; intercepts at least one picture to be recognized from the real-time image; performs recognition on the picture to be recognized and the second environmental information based on a panoramic image recognition model, and outputs a first recognition result; wherein, the training sample of the panoramic image recognition model is the second picture sample obtained by the above-mentioned transmission line icing picture sample generation method based on the diffusion algorithm; performs monitoring and analysis based on the first recognition result.

[0015] Based on the above technical solution, it is possible to generate a large number of second picture samples under the condition of few initial sample pictures, which are used for the training of the panoramic image recognition model. Thus, the panoramic image recognition model can be put into use as soon as possible, avoiding the extension of the system development cycle due to insufficient samples. In addition, through the multi-modal sample input, the recognition accuracy of the panoramic image recognition model can be improved.

[0016] In one embodiment, the method further includes: identifying and extracting target elements and key elements in the picture to be recognized, and correspondingly generating a first recognition picture and a second recognition picture; recognizing the first recognition picture, the second recognition picture and the second environmental information based on the split-scene image recognition model, and outputting a second recognition result; and performing monitoring and analysis based on the first recognition result and the second recognition result.

[0017] Based on the above technical solution, by introducing the split-scene image recognition model to recognize the target elements and key elements in the picture to be recognized, the system can assist in judging the recognition result of the panoramic image recognition model. Especially in the initial application stage when the panoramic image model training is insufficient, the final recognition result can be determined by combining the recognition results of the two, which can improve the recognition accuracy.

[0018] In one embodiment, the intercepting at least one picture to be recognized from the real-time image includes intercepting the image corresponding to at least one key frame from the real-time image as the picture to be recognized.

[0019] In addition, an embodiment of the present application further provides a transmission line monitoring system, which includes a data acquisition device and a data analysis device, and the data analysis device is used to implement the above-mentioned transmission line icing monitoring method.

[0020] An embodiment of the present application further provides an electronic device, which includes a processor, a memory, and a program or instruction stored on the memory and executable on the processor. When the program or instruction is executed by the processor, the above method is implemented.

[0021] In summary, the embodiments of the present application at least include the following beneficial technical effects:

[0022] 1. Realize generating pictures from one picture. Under the condition of insufficient training samples, it is possible to generate multiple training sample pictures based on one initial sample picture, thereby reducing the workload of manual collection and sample annotation.

[0023] 2. To a certain extent, ensure that the newly generated sample pictures are close to the real situation and improve the sample quality.

[0024] 3. The video images of the transmission line are collected by the drone, and the pictures corresponding to the key frames are intercepted from the video images as the pictures to be recognized. This can not only ensure the integrity and clarity of the pictures to be recognized, but also reduce the collection difficulty of the drone, facilitate the rapid completion of the collection work, and reduce the failure rate of the drone. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings forming a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application.

[0026] In order to more clearly illustrate the technical solutions in the embodiments of this application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0027] Figure 1 The flowchart of a method for generating a transmission line icing picture sample based on a diffusion algorithm provided by an embodiment of this application is shown.

[0028] Figure 2 The flowchart of the method for generating pictures from pictures provided by an embodiment of this application is shown.

[0029] Figure 3 The flowchart of a method for generating a transmission line icing picture sample based on a diffusion algorithm provided by another embodiment of this application is shown.

[0030] Figure 4 The flowchart of a transmission line icing monitoring method provided by an embodiment of this application is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0031] The following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the drawings in the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, rather than all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application.

[0032] In the description of the embodiments of this application, unless otherwise specified, "a plurality of" means two or more, and the "first", "second" and various numerical numbers are only for the convenience of description and do not limit the scope of the embodiments of this application.

[0033] The features, structures or characteristics in this application can be combined in one or more embodiments in any suitable manner. In various embodiments of this application, the sequence numbers of the processes do not imply the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of this application.

[0034] Some optional features in the embodiments of this application can, in certain scenarios, be implemented independently without relying on other features, solve corresponding technical problems, achieve corresponding effects, or can also be combined with other features according to requirements in certain scenarios.

[0035] In this application, unless otherwise specified, the same or similar parts between various embodiments can be referred to each other. In various embodiments of this application, if there is no special specification and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be mutually referenced. The technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships. The implementation manners of this application do not constitute a limitation to the protection scope of this application.

[0036] The embodiments of this application are described in detail below with reference to the drawings.

[0037] Please refer to Figure 1 , a method for generating a transmission line icing picture sample based on a diffusion algorithm provided by an embodiment of this application, which is executed by a picture generation server, specifically includes the steps:

[0038] S110, collect the initial sample pictures of the transmission line and the first environmental information.

[0039] In specific implementation, the method for collecting the initial sample pictures of the transmission line may include manual collection based on an image acquisition device, and collection by a drone equipped with an image acquisition device. The obtained initial sample pictures of the transmission line are pictures of the transmission line icing taken in a real scene.

[0040] In one example, the first environmental information includes temperature data and humidity data. The temperature data and humidity data are used to indicate the environment where the icing transmission line is located at that time, and can be the environmental temperature and humidity at the moment when the initial sample pictures are collected. These data can be collected by temperature and humidity sensors when taking the initial sample pictures. The temperature and humidity sensors can be installed on the image acquisition device or directly on the drone. Since they are close to the icing transmission line, they can more truly reflect the environmental conditions of the transmission line.

[0041] S120, identify and extract the target elements and key elements in the initial sample pictures, and label each target element and key element based on the first environmental information to obtain the first picture sample.

[0042] Among them, the target element is the image of an ice-covered transmission line, and the key elements are the environmental elements in the picture background, such as mountains, trees, buildings, fields, rivers, birds, and the sky, etc. In the example, each element in the initial sample picture can be identified and extracted based on the image feature extraction technology to generate independent element pictures, and these element pictures are labeled based on the first environmental information and the identified element names.

[0043] In one implementation, the annotation information of the target element and the key elements further includes ice thickness information, which is used to indicate the ice thickness of the transmission line in the target element. It can be a specific value or a thickness level, and this application is not limited thereto. The acquisition of the thickness information can be obtained by artificial combination with actual experience, or can be calculated based on image processing technology. Preferably, the thickness information can be synchronously detected during the process of collecting the initial sample image.

[0044] It is worth noting that although the key elements do not include the transmission line, the environment where they are located is the same as that of the target element. Therefore, it can be used to indirectly reflect the possible ice thickness of the transmission line in the current environment. Based on this, the thickness information is labeled for the key elements, which is also used to realize the calculation of the ice-covered condition of the transmission line based on the background environmental elements.

[0045] In one implementation, the annotation information of the target element further includes the ice-covered position. In one example, the ice-covered position can be characterized by the distance between the target element and the key component, where the key component can include a tower.

[0046] S130, Write each first picture sample into the model material library and trigger the picture-to-picture process to obtain new sample pictures.

[0047] Specifically, the model material library is used to store the first picture samples obtained through the above steps. These samples can be used to generate new sample pictures. In one implementation, a trigger can be added to the model material library to trigger the picture-to-picture process when new data is written. Based on this, even when there is no data in the model material library before the first picture sample is written, the first picture sample can also be used as the material for this picture generation, thereby realizing fast picture generation.

[0048] Please refer to Figure 2 , The picture-to-picture process method specifically includes:

[0049] S210, Retrieve historical first picture samples from the model material library whose annotation information matches the first environmental information.

[0050] Among them, the annotation information matching the first environmental information includes: the differences between the temperature data and humidity data in the annotation information and the temperature data and humidity data in the first environmental information are all within the corresponding preset ranges.

[0051] Specifically, the temperature preset range and humidity preset range are stored on the image generation server, and these preset ranges are set based on the influence of historical meteorological data on the ice coating thickness. For example, when the temperature is above 0°C, it may be in the ice and snow melting stage, and the ice coating thickness changes relatively fast. Therefore, the temperature preset range value is set to 2°C, that is, if the temperature change range is within 2°C, it is determined that the ice coating situation of the transmission line is roughly the same; when the temperature is below 0°C, the ice and snow melting speed is slow and the change is slow, and the temperature preset range value can be set to 5°C. Similarly, the humidity preset range can be set according to different humidity ranges, and can be specifically determined based on the historical data of the corresponding region.

[0052] Based on this, historical first picture samples that meet the requirements, that is, historical target elements and historical key elements, are retrieved through the preset range, providing effective basic materials for the subsequent picture generation steps, and can improve the rationality of the new pictures. It should be noted that the historical first picture samples are retrieved from the model material library, so the first picture samples are also included in the historical first picture samples.

[0053] S220, construct the picture output requirements based on the target elements, key elements and historical first picture samples.

[0054] In one example, the image generation server can construct output requirements based on the target elements, key elements, and historical target elements or historical key elements included in the historical first picture samples, including generating a new picture based on the target elements and at least one historical key element, or generating a new picture based on the historical target elements and at least one key element, etc.

[0055] S230, process the output requirements based on the image generation model to obtain a new sample picture.

[0056] In one example, the image generation model can be constructed based on the diffusion algorithm. The output requirements are input into the image generation model as prompt words, and the model replaces elements of the initial sample picture based on the diffusion algorithm to obtain a new sample picture.

[0057] In one example, to improve the efficiency of image generation, a pipeline-based image generation method can be implemented based on the ComfyUI workflow. That is, set the folder storing the initial sample images as the source path, and through a script, sequentially read the initial sample images from the source path into the ComfyUI workflow. The image generation model is pre-loaded in the workflow, and based on this, batch generation of new sample images can be achieved, that is, batch output of the newly generated sample images corresponding to all the initial sample images in the folder. The user only needs to deposit the initial sample images into the source path.

[0058] Among them, the ComfyUI workflow is a node-based graphical user interface (GUI) designed specifically for StableDiffusion and used to create complex image generation workflows. By dragging and connecting different nodes, users can build the entire process from loading the model to generating images.

[0059] In one implementation, to enrich the sample style, the newly generated sample images output by the model can also be processed in a workflow based on an image processing workflow, and each image obtained after each step of the processing during the workflow processing is saved as a new sample image.

[0060] In one example, the image processing workflow can be implemented based on ComfyUI. Specifically, an image processing workflow can be built based on ComfyUI. For example, the image is sequentially adjusted for lighting, filtered, and style-converted, etc., and finally a target sample image and intermediate sample images corresponding to each processing node are obtained. These sample images are all saved as new sample images for subsequent operations. In this way, multiple new sample images can be generated based on one initial sample image, greatly enriching the number of samples in the case of fewer initial sample images.

[0061] It should be noted that the above image generation process implemented based on ComfyUI is different from the native functions of ComfyUI. Therefore, it is necessary to implant a custom script in the ComfyUI software to achieve it. For example, by running a script to modify the front-end request mechanism of ComfyUI to automatically scan a specified directory (source path), and automatically construct a processing request based on the newly scanned initial sample images and send it to the back-end service to request processing of the initial sample image based on the workflow. Thus, automatic image reading and processing are achieved, saving a large amount of manual operations.

[0062] In another implementation, the nodes in the workflow can be monitored through a script. When it is monitored that a new image is generated at this node, the image data is copied and output to a specified path for saving. Based on this, the intermediate sample images in the workflow can be output without affecting the processing flow of the workflow.

[0063] Please return to Figure 1 , the method further includes:

[0064] S140, label each new sample picture and the initial sample picture respectively based on the first environmental information to obtain the second picture sample.

[0065] Specifically, the labeling method of the second sample picture is the same as the implementation method in step S120, and the labeled content is also the same, that is, the labeled content of the second picture sample may include icing thickness data, icing position data, etc.

[0066] Based on the above technical solutions, it is possible to generate multiple new sample pictures based on one initial sample picture, and the materials generated from the new sample pictures all come from the actually collected sample pictures, which can improve the effectiveness of the new sample pictures. Further, by selecting the corresponding picture generation basis based on temperature and humidity data, the degree of the new sample being close to the real picture can be further improved, ensuring the quality of the training samples.

[0067] In a preferred embodiment, in order to ensure the quality of the second picture sample, height data can be introduced to screen the new sample pictures to ensure the rationality of the new sample pictures.

[0068] Please refer to Figure 3 , Figure 3 illustrates a method for generating transmission line icing picture samples based on a diffusion algorithm provided by another embodiment of the present application, including the steps:

[0069] S310, collect the initial sample pictures of the transmission line and the first environmental information.

[0070] The difference between this step and the above step S110 is that the first environmental information further includes height data, that is, the altitude at which the image acquisition device takes the transmission line icing image.

[0071] S320, identify and extract the target elements and key elements in the initial sample picture, and label each target element and key element respectively based on the first environmental information to obtain the first picture sample.

[0072] S330, write each first picture sample into the model material library and trigger the picture-to-picture process to obtain new sample pictures.

[0073] S340, screen the new sample pictures based on the height data.

[0074] Specifically, the height data can be used to judge the rationality of key elements. For example, above a certain altitude, key elements should not include shorter objects such as trees and houses. Therefore, the corresponding relationship between height data and key element types can be established in advance. Based on this, new sample pictures can be screened, and the new sample pictures with higher rationality can be retained for subsequent sample generation. Thus, the rationality of the new sample pictures is improved to a certain extent.

[0075] S350, respectively label the screened new sample pictures and the initial sample pictures based on the first environmental information to obtain the second picture samples.

[0076] The operations of the above steps S320, S330, and S350 are basically the same as those of steps S120, S130, and S140, except that the annotation information further includes height data.

[0077] Based on the same inventive concept, an embodiment of the present application also provides a method for monitoring icing on a transmission line, which is applied to a transmission line monitoring system. The system includes a data acquisition device and a data analysis device, and the two interact with each other through a network.

[0078] Please refer to Figure 4 , and this method is executed by the data analysis device, including:

[0079] S410, receive the real-time image of the transmission line to be monitored and the second environmental information sent by the data acquisition device.

[0080] In an example, the data acquisition device is a drone equipped with a camera device and temperature and humidity sensors. The real-time image of the transmission line is collected by remotely controlling the drone, and the real-time image, temperature data, humidity data, and flight altitude data are sent to the data analysis device. It should be noted that in order to ensure the picture quality, the data acquisition device captures a video image.

[0081] S420, intercept at least one picture to be recognized from the real-time image.

[0082] Among them, at least one image corresponding to a key frame can be intercepted from the real-time image as the picture to be recognized. Generally speaking, the image corresponding to the key frame has a clearer picture and is easier to recognize the target elements and key elements therein. Therefore, all the image frames corresponding to the key frames in the real-time image can be directly used as the pictures to be recognized, or several images can be selected therefrom as the pictures to be recognized.

[0083] S430, recognize the picture to be recognized and the second environmental information based on the panoramic image recognition model, and output the first recognition result.

[0084] Among them, the training samples of the panoramic image recognition model are the second image samples obtained by the method for generating transmission line icing picture samples based on the diffusion algorithm provided in the embodiments of the present application. The input layer of the panoramic image recognition model is all the pictures to be recognized collected this time and the second environmental information. The panoramic image recognition model can be implemented based on LateFusion or Transformer and self-attention mechanism, so that the recognition efficiency of the model can be improved through the multi-modal input layer design. The output layer of the panoramic image recognition model can include the icing conditions of the transmission line, such as whether there is icing, the icing thickness, and the icing position.

[0085] S440, perform monitoring and analysis based on the first recognition result.

[0086] In one implementation, when the first recognition result indicates that the transmission line is iced, the data analysis server can instruct the drone to continue to collect images of other segments of the transmission line, identify them, and then construct the icing image of the entire line based on the recognition results of different segments of the same transmission line, establish an icing three-dimensional model, and then combine the second environmental information and meteorological information to predict the duration of icing and the thickness change trend. Add the predicted time data to the icing three-dimensional model to obtain a demonstration animation of the icing three-dimensional model within the predicted time, so as to show the icing change of the transmission line during the predicted period, and combine various warning data to perform fault warning analysis based on the icing three-dimensional model, including the current icing position, icing degree, the change of icing conditions within the predicted time period, and the corresponding warning level. Finally, the warning dynamic picture constructed based on the icing three-dimensional model is visually displayed through the monitor. It should be noted that in order to ensure the real-time monitoring, the icing three-dimensional model is only used to show the icing conditions of the transmission line within a preset distance range (such as within 60 km), so as to reduce the data processing volume and improve the system analysis efficiency. The system can set multiple data collection devices and multiple data analysis devices to monitor different ranges of transmission lines respectively to ensure the real-time monitoring and analysis.

[0087] Furthermore, in order to realize the self-update of the training samples, the data analysis device can determine the acquisition position of the picture to be recognized according to the positioning data of the drone, that is, the actual icing position corresponding to the picture, and verify the icing position in the model recognition result with the actual icing position to determine the recognition accuracy of the model. When the recognition error is large, it is determined as a misrecognition, and a sub-scene image recognition model is introduced for further recognition. At the same time, the picture to be recognized is stored as a new initial sample picture in the source path, so as to realize the automatic collection of sample data, thereby being used to realize the continuous training of the image recognition model.

[0088] As described above, in practical applications, due to the less-than-ideal initial recognition accuracy of the panoramic image recognition model, or the reason of snowy or icy weather, the collected images may not be clear enough, resulting in misrecognition or non-recognition of the images to be recognized. Based on this, the sub-scene image recognition model can be introduced to recognize the images to be recognized again.

[0089] In one embodiment, the above method further includes: recognizing and extracting the target elements and key elements in the image to be recognized, and correspondingly generating a first recognition image and a second recognition image; recognizing the first recognition image, the second recognition image and the second environmental information based on the sub-scene image recognition model, and outputting a second recognition result; and performing monitoring and analysis based on the first recognition result and the second recognition result. Among them, the sub-scene image recognition model is trained based on the first image samples in the model material library.

[0090] Based on the sub-scene image recognition model, even when only part of the image to be recognized is clear, the recognition result can be obtained according to the recognition of the elements. Especially in the initial stage of system use, due to the lack of rich sample data, the recognition accuracy of the panoramic image recognition model may not be high. Therefore, by combining the sub-scene image recognition model, the verification or correction of the first recognition result can be realized. For example, when the two results are the same, the confidence level of the panoramic image recognition model can be increased. If they are different, the result data can be fed back to the user, and new training samples can be constructed based on the suggestions fed back by the user for the optimization training of the panoramic image recognition model and the sub-scene image recognition model, so as to continuously improve the recognition accuracy of the model.

[0091] It should be noted that since the recognition and processing process of the sub-scene image recognition model is relatively complex and is more limited by image recognition technology, in some embodiments of the present application, the sub-scene image recognition model is only used for auxiliary verification when the recognition accuracy of the panoramic image recognition model is not ideal. When the recognition accuracy (i.e., confidence level) of the panoramic image recognition model meets the preset target, there is no need to use the sub-scene image recognition model in the process of image recognition, thus saving system computing resources.

[0092] It can be understood that in the case of fewer initial sample images and insufficient model training, the reason why the recognition accuracy of the sub-scene image recognition model is higher than that of the panoramic image recognition model is that: the range of the first image samples is wider than that of the second image samples, the difference degree between the samples is larger, and the recognition objects include target elements and key elements. Even if some of the elements are not clear enough, the result can be output based on the recognition results of other elements. Therefore, in the early stage of system use, the sub-scene image recognition model can be introduced to verify the panoramic image recognition model and judge the confidence level, which can improve the image recognition accuracy in the initial stage of application and ensure the normal use of the system.

[0093] In addition, an embodiment of the present application further provides an electronic device, which includes a processor, a memory, and a program or instruction stored on the memory and executable on the processor. When the program or instruction is executed by the processor, the method in any implementation manner in the embodiment of the present application is implemented; among them, the processor may adopt a general central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processing unit (GPU), or one or more integrated circuits, and is used to execute relevant programs to implement the method in any implementation manner in the embodiment of the present application.

[0094] The processor may also be an integrated circuit electronic device with the ability to process signals. In the implementation process, each step of the method in any implementation manner in the embodiment of the present application may be completed by the integrated logic circuit in the hardware of the processor or the instruction in the form of software.

[0095] The above-mentioned processor may also be a general-purpose processor, a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiment of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiment of the present application may be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by the combination of the hardware and software modules in the decoding processor.

[0096] The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory. The processor reads the information in the memory and combines its hardware to complete the functions required to be executed by the units included in the data processing device of the embodiment of the present application, or executes the method in any implementation manner in the embodiment of the present application.

[0097] The above are all the preferred embodiments of the present application. The protection scope of the present application is not limited accordingly. Therefore, all equivalent changes made according to the structure, shape, and principle of the present application should be covered within the protection scope of the present application.

Claims

1. A method for generating ice-covered image samples of power transmission lines based on a diffusion algorithm, characterized in that: The method comprises the steps of: Collecting initial sample images of transmission lines and first environmental information; Identify and extract target elements and key elements in the initial sample image, and mark each of the target elements and the key elements based on the first environmental information to obtain a first image sample; the target element is an image of an ice-covered power transmission line, and the key element is an environmental element in the image background; Each of the first picture samples is written into the model material library and the image generation process is triggered to obtain a new sample picture; wherein the image generation process includes retrieving the historical first picture samples whose annotation information matches the first environmental information from the model material library; constructing picture output requirements based on the target element, the key element, and the historical first picture samples; processing the output requirements based on the image generation model to obtain the new sample picture, wherein the image generation model is implemented based on a diffusion algorithm; Based on the first environmental information, each of the new sample pictures and the initial sample picture is labeled to obtain a second picture sample.

2. The method according to claim 1, characterized in that The image generation process is implemented based on the workflow of ComfyUI.

3. The method according to claim 2, characterized in that The first environmental information includes temperature data and humidity data; the tag information matches the first environmental information including: the difference between the temperature data and humidity data in the tag information and the corresponding data in the first environmental information are both within a corresponding preset range.

4. The method according to claim 3, characterized in that The temperature data and humidity data are collected by sensors installed on the drone when taking the initial sample pictures.

5. The method according to claim 1, characterized in that The first environmental information includes altitude data, and the method further includes: screening the new sample image based on the altitude data.

6. A method for monitoring ice coating on a power transmission line, characterized in that: Applied to a power transmission line monitoring system, the system includes a data acquisition device and a data analysis device, and the method includes: The data analysis device receives the real-time image of the power transmission line to be monitored and the second environmental information sent by the data acquisition device; Intercepting at least one picture to be identified from the real-time image; The image to be identified and the second environmental information are identified based on a panoramic image recognition model, and a first recognition result is output; wherein the training sample of the panoramic image recognition model is the second image sample obtained by the method for generating an ice-covered image sample of a power transmission line based on a diffusion algorithm according to any one of claims 1 to 5; Monitoring and analysis are performed based on the first identification result.

7. The method according to claim 6, characterized in that The method further comprises: Identify and extract target elements and key elements in the image to be identified, and generate a first identification image and a second identification image accordingly; Identify the first identification picture, the second identification picture and the second environment information based on the scene segmentation image recognition model, and output a second identification result; Monitoring and analysis are performed based on the first recognition result and the second recognition result.

8. The method according to claim 6, characterized in that The capturing at least one picture to be identified from the real-time image includes capturing an image corresponding to at least one key frame from the real-time image as the picture to be identified.

9. A power transmission line monitoring system, characterized in that: The system comprises a data acquisition device and a data analysis device, and the data analysis device is used to implement the method according to any one of claims 6 to 8.

10. An electronic device, characterized in that: The electronic device includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction implements the method according to any one of claims 1 to 8 when executed by the processor.

Images