Image generation method and system based on substation equipment defects

Abstract

The invention provides an image generation method and system based on substation equipment defects, electronic equipment and a computer readable storage medium. The method comprises the steps of obtaining an original substation equipment defect text; generating a transformer substation equipment defect basic image based on the original transformer substation equipment defect text by using a diffusion model; utilizing a diffusion model to generate a transformer substation equipment defect image according to the transformer substation equipment defect basic image and the fine-grained transformer substation equipment defect text; wherein the fine-grained substation equipment defect text is obtained by performing text enhancement on the original substation equipment defect text through a text enhancement model based on a Decoder-only architecture. According to the method, the diffusion model framework including two-stage image generation is arranged, so that the technical effect of improving the synthesis accuracy and quality of the text-driven substation equipment defect image is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and substation technology, and specifically relates to an image generation method, system, electronic device and computer-readable storage medium based on substation equipment defects. Background Art

[0002] As a key component of the power system, the safe and stable operation of substation equipment plays a vital role in ensuring the safety of the power grid. Traditional inspection methods can no longer meet the current needs for operation and maintenance and safety monitoring. Therefore, intelligent inspection technology has emerged and become an important means to improve inspection efficiency. Intelligent inspection is mainly based on deep learning algorithms, which require a large amount of data for model training and iterative learning. However, public resources for substation equipment defect data sets are scarce, and it is difficult to collect real defect data. In particular, some equipment defects have a low frequency of occurrence, but once they occur, they are highly dangerous and may cause economic losses and casualties to substations. Therefore, how to generate training data with a high degree of simulation, a complete range of component types, and rich and realistic defect forms has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present invention provides an image generation method, system, electronic device and computer-readable storage medium based on substation equipment defects, so as to overcome at least one technical problem existing in the prior art.

[0004] To achieve the above object, the present invention provides an image generation method based on substation equipment defects, the method comprising:

[0005] Get the original substation equipment defect text;

[0006] Generate a basic image of substation equipment defects based on the original substation equipment defect text using a diffusion model;

[0007] Generate a substation equipment defect image according to the substation equipment defect basic image and fine-grained substation equipment defect text by using a diffusion model;

[0008] The fine-grained substation equipment defect text is obtained by performing text enhancement on the original substation equipment defect text through a text enhancement model based on a decoder-only architecture.

[0009] Further, the preferred method is:

[0010] The method for obtaining the fine-grained substation equipment defect text by performing text enhancement on the original substation equipment defect text through a text enhancement model based on a decoder-only architecture comprises:

[0011] The original substation equipment defect text is converted into a token sequence through the SentencePiece model, and the token sequence is converted into a high-dimensional vector representation using the embedding layer;

[0012] The high-dimensional vector representation is processed by a decoding layer and a fine-grained substation equipment defect text is output.

[0013] Further, the preferred method is:

[0014] The decoding layer includes a self-attention mechanism and a feed-forward neural network.

[0015] Further, the preferred method is:

[0016] The method for generating a basic image of substation equipment defects based on the original substation equipment defect text by using a diffusion model comprises:

[0017] Encoding the original substation equipment defect text into a high-dimensional text feature vector through the CLIP model, and mapping the encoded high-dimensional text feature vector to a latent space;

[0018] In the latent space, the predicted image of the initial state is subjected to reverse denoising iteration according to a preset step size threshold until a latent image matching the original substation equipment defect text is generated;

[0019] The potential image is used as a basic image of substation equipment defects.

[0020] Further, the preferred method is:

[0021] The preset step length threshold is t, 700>t>300.

[0022] Further, the preferred method is:

[0023] After generating the substation equipment defect image, the method further includes performing substation equipment defect detection using the substation equipment defect image.

[0024] Further, the preferred method is:

[0025] The method for generating a substation equipment defect image by using a diffusion model according to the substation equipment defect basic image and fine-grained substation equipment defect text comprises:

[0026] The fine-grained substation equipment defect text is encoded into a high-dimensional text feature vector through the CLIP model, and the substation equipment defect basic image is encoded into a high-dimensional image feature vector; and the encoded high-dimensional text feature vector and high-dimensional image feature vector are mapped to a latent space to obtain a predicted image of an initial state;

[0027] Using the encoder of U-Net to gradually downsample the input image to reduce the receptive field in the diffusion process, and reversely denoising the predicted image in the initial state according to a preset step threshold until a potential image matching the fine-grained substation equipment defect text is generated;

[0028] The potential image is used as a substation equipment defect image.

[0029] In order to solve the above problems, the present invention also provides an image generation system based on substation equipment defects, which uses the above-mentioned image generation method based on substation equipment defects to generate images. The system includes:

[0030] A text acquisition unit, used to acquire original substation equipment defect text;

[0031] A defect basic image generating unit, used for generating a substation equipment defect basic image based on the original substation equipment defect text by using a diffusion model;

[0032] A defect image generation unit is used to generate a substation equipment defect image based on the substation equipment defect basic image and fine-grained substation equipment defect text using a diffusion model; wherein the fine-grained substation equipment defect text is obtained by performing text enhancement on the original substation equipment defect text through a text enhancement model based on a decoder-only architecture.

[0033] In order to solve the above problems, the present invention also provides an electronic device, which includes a memory, a processor, and an image generation program based on substation equipment defects stored in the memory and executable on the processor, wherein the image generation program based on substation equipment defects implements the steps in the above-mentioned image generation method based on substation equipment defects when executed by the processor.

[0034] In order to solve the above problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned image generation method based on substation equipment defects.

[0035] The present invention provides an image generation method, system, electronic device and computer-readable storage medium based on substation equipment defects, which aims to improve the accuracy and quality of text-driven synthesis of substation equipment defect images by proposing a two-stage generation model framework. The first stage is rough image draft generation; a rough image draft is generated directly based on the original text description. This step quickly captures the main visual elements in the text and lays the foundation for subsequent refinement. The second stage is fine-grained text enhancement and image refinement; using the fine-grained text enhancement module, more detailed and specific information is extracted from the original text to generate a higher quality text description. At the same time, by adjusting the receptive field in the diffusion model, the model can process and improve the image draft generated in the first stage at a more detailed level. This step involves fine-tuning the local features of the image to ensure that the generated image remains stable in global contour and texture, while achieving realistic technical effects in details. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A schematic flow chart of a method for generating an image based on substation equipment defects provided by an embodiment of the present invention.

[0038] Figure 2 A schematic diagram of the principle of a method for generating an image based on substation equipment defects provided by an embodiment of the present invention.

[0039] Figure 3 A schematic diagram of the principle of the first stage of a method for generating images based on substation equipment defects provided by an embodiment of the present invention.

[0040] Figure 4 A schematic diagram of the principle of the second stage of the method for generating images based on substation equipment defects provided by an embodiment of the present invention.

[0041] Figure 5 A schematic diagram of the principles of a text enhancement model based on a decoder-only architecture provided in one embodiment of the present invention.

[0042] Figure 6 A schematic diagram of modules of an image generation system based on substation equipment defects provided by an embodiment of the present invention.

[0043] Figure 7A schematic diagram of the internal structure of an electronic device for implementing an image generation method based on substation equipment defects provided by an embodiment of the present invention.

[0044] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0045] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0046] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, "and / or" in the text is only a description of the association relationship of the associated objects, indicating that three relationships may exist, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone.

[0047] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In addition, in the description of the embodiments of the present application, "plurality" means two or more than two.

[0048] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0049] Diffusion Models: Diffusion model is a generative model based on probability theory, which was originally derived from the theory of diffusion process in physics, such as the diffusion process of ink in water. In the field of machine learning, this concept has been creatively applied to data generation tasks, especially the synthesis of images and sounds. By simulating a gradual "diffusion" process from data distribution to simple noise distribution, and then learning the inverse process to reconstruct high-quality data samples from the noise. Among them, StableDifiusion: A deep learning model for generating images, which uses a technique called diffusion model. This model can generate images based on text descriptions, and can also perform style transfer or repair on existing images.

[0050] CLIP: (Contrastive Language-Image Pre-training) is a multimodal pre-trained neural network launched by OpenAI in 2021. It uses a large amount of paired data of images and texts for pre-training through contrastive learning to learn the alignment relationship between images and texts. The CLIP model has two modalities, one is the text modality and the other is the visual modality, and includes two main parts: Text Encoder is used to convert text into a low-dimensional vector representation (Embedding), and Image Encoder is used to convert images into similar vector representations (Embedding). In the prediction stage, the CLIP model generates predictions by calculating the cosine similarity between text and image vectors, which is particularly suitable for zero-shot learning tasks, that is, the model does not need to see new training examples of images or text to make predictions.

[0051] The decoder-only architecture is a neural network model architecture specifically designed for natural language processing (NLP) tasks. Compared to the common encoder-decoder architecture, the decoder-only architecture only has a decoder part and no encoder. An important feature of the decoder-only architecture is that it can be pre-trained without supervision. During the pre-training phase, the model learns the statistical patterns and semantic information of the language through a large amount of unlabeled data. This pre-training method can enable the model to have extensive language knowledge and understanding capabilities. After pre-training, the model can be fine-tuned in a supervised manner for specific downstream tasks (such as machine translation, text generation, etc.).

[0052] U-Net is one of the earliest algorithms that used a fully convolutional network for semantic segmentation. The use of a symmetrical U-shaped structure containing a compression path and an expansion path was very innovative at the time and influenced the design of several subsequent segmentation networks to a certain extent. The name of the network is also taken from its U-shaped shape.

[0053] Example 1

[0054] Figure 1 The image generation method based on substation equipment defects is described in general. Figure 1 A flowchart of a method for generating an image based on substation equipment defects provided by an embodiment of the present invention; the method may be executed by a system, which may be implemented by software and / or hardware.

[0055] like Figure 1 As shown, in this embodiment, the image generation method based on substation equipment defects includes steps S110 to S130.

[0056] S110. Obtain original substation equipment defect text.

[0057] A database of substation equipment defects can be prepared based on actual substation equipment maintenance experience, which can include, for example, aging of voltage transformer insulation, contamination of insulator surface, and damage to cable sheaths.

[0058] S120, using a diffusion model to generate a substation equipment defect basic image based on the original substation equipment defect text. S130, using a diffusion model to generate a substation equipment defect image based on the substation equipment defect basic image and fine-grained substation equipment defect text; wherein the fine-grained substation equipment defect text is obtained by text enhancement of the original substation equipment defect text by a text enhancement model based on a decoder-only architecture.

[0059] Figure 2-Figure 5 The image generation method based on substation equipment defects is described in general. Figure 2 A schematic diagram of the principle of a method for generating an image based on substation equipment defects provided by an embodiment of the present invention. Figure 3 A schematic diagram of the principle of the first stage of a method for generating images based on substation equipment defects provided by an embodiment of the present invention. Figure 4 A schematic diagram of the principle of the second stage of the method for generating images based on substation equipment defects provided by an embodiment of the present invention. Figure 5 A schematic diagram of the principles of a text enhancement model based on a decoder-only architecture provided in one embodiment of the present invention.

[0060] like Figure 2As shown, the present invention proposes a two-stage generation model framework to improve the accuracy and quality of text-driven substation equipment defect image synthesis. Among them, the first stage is the generation of a rough image draft; a rough image draft is generated directly according to the original text description. This step quickly captures the main visual elements in the text and lays the foundation for subsequent refinement. Specifically, for the first stage, during the input process, the model receives the original text description (original substation equipment defect text encoding) input by the user, that is, a simplified component and defect description. During the processing, the original substation equipment defect text is first encoded by the CLIP model, and then input into the Stable Diffusion model (diffusion model), and after the first stage of t-step denoising in the large receptive field, a preliminary image draft is directly generated according to the text description. During the output process, the generated preliminary image should contain the main visual elements in the text description, although the details may not be fine enough. In short, in the first stage of the model, the goal is to convert the original text description provided by the user into a basic visual representation, laying the foundation for the subsequent fine-grained enhancement stage.

[0061] The second stage is fine-grained text enhancement and image refinement; the fine-grained text enhancement module is used to extract more detailed and specific information from the original text to generate a higher quality text description. At the same time, by adjusting the receptive field in the diffusion model, the model can process and improve the image draft generated in the first stage at a more detailed level. Specifically, for the second stage, during the input process, the text processed by fine-grained enhancement (fine-grained substation equipment defect text), that is, detailed parts and defect descriptions. And the coarse-grained image generated in the first stage are used as input. During the processing, the fine-grained substation equipment defect text is first encoded by the CLIP model, and then input into the Stable Diffusion model (diffusion model), and denoised by t steps in the second stage in a small receptive field. The Stable Diffusion model (diffusion model) is used again to refine and enhance the image in combination with more detailed text information and preliminary images. During the output process: the final image generated should have higher quality and more accurately reflect the intention and details of the user's original text description.

[0062] In summary, the present invention optimizes the existing stable diffusion model and proposes a two-stage generative model framework to enhance the accuracy and quality of text-driven image synthesis. In the first stage, the original text is directly used to generate a rough image draft; in the second stage, the fine-grained text enhancement module is used to generate more detailed text information, and the generated draft is carefully processed and improved while reducing the receptive field of the diffusion process. In this way, defective images of substation equipment with stable global contours and textures and realistic details are generated.

[0063] The goal of the first stage is to use the original text description to directly generate a latent image with basic structure and contour. In this stage, the text description does not go through any fine-grained enhancement module, but is directly CLIP encoded and input into the model as a text condition. The design intention is to quickly capture the main visual elements of the text description in the first stage and form a preliminary image sketch, laying the foundation for subsequent detail filling. Figure 3 As shown, the method for generating a basic image of substation equipment defects by using a diffusion model based on the original substation equipment defect text includes steps S121 to S123.

[0064] S121. Encode the original substation equipment defect text into a high-dimensional text feature vector through the CLIP model, and map the encoded high-dimensional text feature vector to a latent space.

[0065] Specifically, the original text description provided by the user is used as the model input. The CLIP model is used to convert the text description into a high-dimensional text feature vector. This step is critical because the CLIP model is trained on a large number of image-text pairs and can capture rich semantic information to help guide image generation.

[0066] S122. Perform reverse denoising iterations on the predicted image of the initial state in the latent space according to a preset step size threshold until a latent image matching the original substation equipment defect text is generated.

[0067] Specifically, the model starts with a high-dimensional noise distribution, which represents the latent space of the image. The model gradually recovers the image from the noise through t steps of inverse denoising. At each step, the output of the previous step and the CLIP-encoded text feature vector are used as conditional inputs to guide the direction of image generation.

[0068] S123: Using the potential image as a basic image of substation equipment defects.

[0069] After t steps of denoising, the resulting latent image Z T-t It contains the basic visual elements and outlines of the original text description, but the details may not be fully clear or refined. It serves as the basic image of substation equipment defects and the input image of the second stage.

[0070] The first stage effectively transforms text descriptions into images with basic visual structures, laying a solid foundation for fine-grained text completion and detail enhancement in the second stage. This staged approach not only speeds up the generation of preliminary images, but also provides more flexibility and possibilities for the refinement of details.

[0071] The second stage aims to further refine and enhance the details of the image based on the text processed by the text enhancement module. Adding the second stage of the fine-grained text enhancement module can not only improve the quality and detail richness of the generated image, but also make the generated image closer to the user's fine-grained text description. Figure 4 As shown, the method for generating a substation equipment defect image by using a diffusion model according to the substation equipment defect basic image and fine-grained substation equipment defect text includes steps S131 to S133.

[0072] S131. Encode the fine-grained substation equipment defect text into a high-dimensional text feature vector and encode the substation equipment defect basic image into a high-dimensional image feature vector through the CLIP model; and map the encoded high-dimensional text feature vector and high-dimensional image feature vector to a latent space to obtain a predicted image of an initial state.

[0073] S132, using the encoder of U-Net to gradually downsample the input image to reduce the receptive field during the diffusion process, and reversely denoising the predicted image of the initial state according to a preset step threshold until a potential image matching the fine-grained substation equipment defect text is generated. Among them, the encoder part of U-Net achieves the reduction of the receptive field by gradually downsampling the input image. This process usually involves convolutional layers and pooling layers to extract high-level features of the image by reducing the spatial resolution of the image. Downsampling allows the network to focus on increasingly abstract representations while reducing the amount of computation and the number of parameters.

[0074] Specifically, the enhanced text after fine-grained processing and the latent image Z generated in the first stage are T–t As the input of the second stage. Specifically, the original text related to the output image of the first stage is input into the fine-grained text enhancement model. The model completes and enhances the original text in a fine-grained manner to generate text with more detailed descriptions and specific details. The fine-grained processed text and the potential image generated in the first stage are used as input conditions and integrated into the image generation model. This step is key and directly links the detailed description of the text to the image generation process. In the second stage, the model continues the reverse denoising process, but at this time it focuses more on enhancing the details of the image based on the fine-grained text. This may involve adjusting the texture, color, lighting effects, and visual semantics of the image to more accurately reflect the content of the text description. The text feature vector encoded by the CLIP model is used as the conditional input, combined with Z T-t-l The feature vector undergoes the final denoising step, and then generates the Z0 vector, which is then decoded by VAE to output the pixel image D.

[0075] It should be noted that this includes but is not limited to: adjusting the texture and details of the image to match the features described in the enhanced text, such as refining "blue silicone tube" to "blue silicone tube with slight cracks". Adjusting the color and light and shadow effects of the image to enhance visual realism and depth, such as simulating the concave and convex feeling of the object surface by adding light and shadow effects. Correcting the visual semantics to ensure that the elements in the image are consistent with the description in the enhanced text, such as changing "cracked silicone tube" to "cracked blue silicone tube, with a clearly visible crack on the left edge of the tube, and the crack length is about 2 / 3 of the silicone tube".

[0076] S133: Use the potential image as a substation equipment defect image.

[0077] The model outputs the final image, which not only has the basic outline and structure formed in the first stage, but also contains rich details and depth through the processing of the second stage, and more accurately reflects the description of the fine-grained text. Through this design, the second stage effectively fuses the latent image with the semantics of the general outline of the object generated in the first stage with the added fine-grained text, which can significantly improve the quality of image generation, so that the generated image not only retains the main content of the original text description, but also meticulously expresses the fine-grained visual content generated by the text enhancement module.

[0078] It should be noted that in the design of the two-stage model, it is crucial to determine the ideal time point t (denoising step size) to transition from the first stage to the second stage. This decision needs to take into account multiple factors, including the quality of the image produced in the first stage, the specific implementation details of the model, and the expectations for the level of image detail. The ideal time point t should be chosen when the image generated in the first stage already has a basic outline, but there is still enough space to further refine and improve the details based on fine-grained text.

[0079] When t is between 300 and 700: At this stage, the image has begun to form basic outlines and some preliminary details, but it is still relatively fuzzy overall. Choosing a t value within this range can retain the general structure generated in the first stage while using fine-grained text to guide the generation of finer details, such as texture, light and shadow effects, and specific features of objects.

[0080] Slightly later stage (t between 700 and 1000): If the image quality generated in the first stage is high and already contains clearer contours and some details, then a slightly later time step can be selected for fine-grained adjustment. This helps to refine the image without destroying the existing basic structure. In the specific implementation process, the preset step size threshold is t, 700>t>300.

[0081] Specifically, the text enhancement module is a text fine-grained model designed for substation defect description based on substation vertical field data. After the rough text is input into the model, a more detailed and fine-grained text description can be generated. Input a short text description, such as "cracked silicone tube", and the model outputs a more detailed description: "cracked blue silicone tube, there is a clearly visible crack on the left edge of the tube, and the length of the crack is about 2 / 3 of the silicone tube."

[0082] like Figure 5 As shown, different from the traditional transformer encoder-decoder architecture text generation model, the text enhancement model proposed in this scheme is based on the Decoder-only architecture, which discards the encoding part in the transformer architecture. A transformer block only contains self-attention and feedforward neural networks, and does not require the participation of encoder-decoder attention. It adopts the idea of ​​autoregression and requires less video memory and computing resources than the traditional architecture in both training and reasoning. Specifically, the input raw text is first converted into a token sequence using the SentencePiece model. This means that the text is first segmented into subword or character-level tokens, which helps to process unknown or rare words. After word segmentation, each token is mapped to a unique integer ID, which will be used to find the embedding vector. Use the integer ID of each token as an index to retrieve the corresponding embedding vector from the embedding layer. The embedding layer is a trainable weight matrix with a dimension of

[0083] [vocab_size, embedding_dim], where vocab_size is the size of the vocabulary and embedding_dim is the dimension of the embedding vector. At this point, the original input text becomes a high-dimensional vector with a dimension of [sequence length, embedding_dim]. This vector passes through N*decoder-transformer-block and then undergoes linear projection to obtain the logits value corresponding to each element in the sequence. Then, it passes through the softmax function to obtain the probability distribution of each element in the sequence belonging to a certain vocabulary. The word corresponding to the maximum probability value is taken as the output to obtain the final generated text.

[0084] In a specific implementation process, the method for obtaining the fine-grained substation equipment defect text by performing text enhancement on the original substation equipment defect text through a text enhancement model based on a decoder-only architecture includes S1301 to S1302.

[0085] S1301. Convert the original substation equipment defect text into a token sequence through the SentencePiece model, and convert the token sequence into a high-dimensional vector representation using an embedding layer.

[0086] S1302: Process the high-dimensional vector representation through a decoding layer and output a fine-grained substation equipment defect text. The decoding layer includes a self-attention mechanism and a feedforward neural network.

[0087] The fine-grained text enhancement model is used to deeply analyze and complete the original text to generate enhanced text with richer semantic information and details. The model can identify and supplement the missing visual elements and specific features in the text, such as enhancing from "car" to "blue car" or from "21" to "bus No. 21".

[0088] In the specific implementation process, the substation equipment defect image is used to detect substation equipment defects. Specifically, a target detection algorithm based on deep learning, such as Faster R-CNN, is used to analyze the generated equipment defect image, automatically frame the equipment area or search for the location where the equipment abnormality occurs, and realize intelligent fault detection.

[0089] like Figure 6 As shown, the present invention provides an image generation system 600 based on substation equipment defects, which can be installed in an electronic device. According to the implemented functions, the image generation system 600 based on substation equipment defects may include a text acquisition unit 610, a defect base image generation unit 620, and a defect image generation unit 630. The unit of the present invention may also be referred to as a module, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0090] In this embodiment, the functions of each module / unit are as follows:

[0091] A text acquisition unit 610, used to acquire original substation equipment defect text;

[0092] A defect basic image generating unit 620 is used to generate a substation equipment defect basic image based on the original substation equipment defect text by using a diffusion model;

[0093] The defect image generation unit 630 is used to generate a substation equipment defect image based on the substation equipment defect basic image and the fine-grained substation equipment defect text using a diffusion model; wherein the fine-grained substation equipment defect text is obtained by performing text enhancement on the original substation equipment defect text using a text enhancement model based on a decoder-only architecture.

[0094] The image generation system 600 based on substation equipment defects of the present invention proposes a two-stage generation model framework to improve the accuracy and quality of text-driven synthesis of substation equipment defect images. Among them, the first stage is the generation of a rough image draft; a rough image draft is generated directly based on the original text description. This step quickly captures the main visual elements in the text and lays the foundation for subsequent refinement. The second stage is fine-grained text enhancement and image refinement; using the fine-grained text enhancement module, more detailed and specific information is extracted from the original text to generate a higher quality text description. At the same time, by adjusting the receptive field in the diffusion model, the model can process and improve the image draft generated in the first stage at a more detailed level. This step involves fine-tuning the local features of the image to ensure that the generated image remains stable in global contour and texture, while achieving realistic technical effects in details.

[0095] like Figure 7 As shown, the present invention also provides an electronic device 1 based on the image generation method of substation equipment defects.

[0096] The electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as an image generation program 12 based on substation equipment defects. The memory 11 may also include both an internal storage unit of the image generation system based on substation equipment defects and an external storage device. The memory 11 may be used not only to store the code of the image generation program based on substation equipment defects installed in the application software and various data, such as the code of the image generation program based on substation equipment defects, but also to temporarily store the data that has been output or is to be output.

[0097] The electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as an image generation program 12 based on substation equipment defects. The memory 11 may also include both an internal storage unit of the image generation system based on substation equipment defects and an external storage device. The memory 11 may be used not only to store the code of the image generation program based on substation equipment defects installed in the application software and various data, such as the code of the image generation program based on substation equipment defects, but also to temporarily store the data that has been output or is to be output.

[0098] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 may not only be used to store application software and various types of data installed in the electronic device 1, such as the code of an image generation program based on defects in substation equipment, but may also be used to temporarily store data that has been output or is to be output.

[0099] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (for example, an image generation program based on substation equipment defects, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device 1 and process data.

[0100] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0101] Figure 7 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 7The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0102] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management system, so that the power management system can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0103] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0104] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0105] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0106] The image generation program 12 based on substation equipment defects stored in the memory 11 in the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve: S110, obtaining the original substation equipment defect text. S120, using the diffusion model to generate a substation equipment defect basic image based on the original substation equipment defect text. S130, using the diffusion model to generate a substation equipment defect image based on the substation equipment defect basic image and the fine-grained substation equipment defect text; wherein the fine-grained substation equipment defect text is obtained by text enhancement of the original substation equipment defect text by a text enhancement model based on a decoder-only architecture.

[0107] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiment is not repeated here. Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or system capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0108] An embodiment of the present invention also provides a computer-readable storage medium, which may be non-volatile or volatile, and stores a computer program, which, when executed by a processor, implements: S110, obtaining the original substation equipment defect text. S120, generating a substation equipment defect basic image based on the original substation equipment defect text using a diffusion model. S130, generating a substation equipment defect image based on the substation equipment defect basic image and fine-grained substation equipment defect text using a diffusion model; wherein the fine-grained substation equipment defect text is obtained by performing text enhancement on the original substation equipment defect text using a text enhancement model based on a decoder-only architecture.

[0109] Specifically, the specific implementation method when the computer program is executed by the processor can refer to the description of the relevant steps in the image generation method based on substation equipment defects in the embodiment, which will not be repeated here.

[0110] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0111] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0112] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0113] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0114] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0115] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or systems stated in a system claim can also be implemented by one unit or system through software or hardware. Secondary words are used to indicate names, but not to indicate any particular order.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for generating an image based on substation equipment defects, characterized in that: Methods include, Get the original substation equipment defect text; Generate a basic image of substation equipment defects based on the original substation equipment defect text using a diffusion model; Generate a substation equipment defect image according to the substation equipment defect basic image and fine-grained substation equipment defect text by using a diffusion model; The fine-grained substation equipment defect text is obtained by performing text enhancement on the original substation equipment defect text through a text enhancement model based on a decoder-only architecture.

2. The image generation method based on substation equipment defects according to claim 1 is characterized in that: The method for obtaining the fine-grained substation equipment defect text by performing text enhancement on the original substation equipment defect text through a text enhancement model based on a decoder-only architecture comprises: The original substation equipment defect text is converted into a token sequence through the SentencePiece model, and the token sequence is converted into a high-dimensional vector representation using the embedding layer; The high-dimensional vector representation is processed by a decoding layer and a fine-grained substation equipment defect text is output.

3. The image generation method based on substation equipment defects according to claim 2 is characterized in that: The decoding layer includes a self-attention mechanism and a feed-forward neural network.

4. The image generation method based on substation equipment defects according to claim 1 is characterized in that: The method for generating a basic image of substation equipment defects based on the original substation equipment defect text by using a diffusion model comprises: Encoding the original substation equipment defect text into a high-dimensional text feature vector through the CLIP model, and mapping the encoded high-dimensional text feature vector to a latent space; In the latent space, the predicted image of the initial state is subjected to reverse denoising iteration according to a preset step size threshold until a latent image matching the original substation equipment defect text is generated; The potential image is used as a basic image of substation equipment defects.

5. The image generation method based on substation equipment defects according to claim 1 is characterized in that: The preset step length threshold is t, 700>t>300.

6. The image generation method based on substation equipment defects according to claim 1 is characterized in that: After generating the substation equipment defect image, the method further includes performing substation equipment defect detection using the substation equipment defect image.

7. The image generation method based on substation equipment defects according to claim 1 is characterized in that: The method for generating a substation equipment defect image by using a diffusion model according to the substation equipment defect basic image and fine-grained substation equipment defect text comprises: The fine-grained substation equipment defect text is encoded into a high-dimensional text feature vector through the CLIP model, and the substation equipment defect basic image is encoded into a high-dimensional image feature vector; and the encoded high-dimensional text feature vector and high-dimensional image feature vector are mapped to a latent space to obtain a predicted image of an initial state; Using the encoder of U-Net to gradually downsample the input image to reduce the receptive field in the diffusion process, and reversely denoising the predicted image in the initial state according to a preset step threshold until a potential image matching the fine-grained substation equipment defect text is generated; The potential image is used as a substation equipment defect image.

8. An image generation system based on substation equipment defects, characterized in that: The image generation method based on substation equipment defects according to any one of claims 1 to 7 is used to generate an image, and the system includes: A text acquisition unit, used to acquire original substation equipment defect text; A defect basic image generating unit, used for generating a substation equipment defect basic image based on the original substation equipment defect text by using a diffusion model; A defect image generation unit is used to generate a substation equipment defect image based on the substation equipment defect basic image and fine-grained substation equipment defect text using a diffusion model; wherein the fine-grained substation equipment defect text is obtained by performing text enhancement on the original substation equipment defect text through a text enhancement model based on a decoder-only architecture.

9. An electronic device, characterized in that: The electronic device includes a memory, a processor, and an image generation program based on substation equipment defects stored in the memory and executable on the processor. When the image generation program based on substation equipment defects is executed by the processor, the steps of the image generation method based on substation equipment defects as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the image generation method based on substation equipment defects as described in any one of claims 1 to 7 is implemented.

Images