Image domain expansion method and system based on domain style
By employing a domain-style-based image domain expansion method, high-quality domain-expanded images are generated using neural convolutional networks and loss functions. This addresses the issues of data shortage and domain transformation, enhances the model's generalization ability, and improves the accuracy of insulator detection.
Patent Information
- Application Number
- CN202511046500.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-29
- Publication Date
- 2025-10-31
AI Technical Summary
In power defect detection, data shortages and domain transformation issues lead to insufficient model generalization ability. Existing methods cannot effectively expand the dataset, especially in insulator detection where defect data is limited and the diversity of detection scenarios is insufficient.
We employ a domain-style-based image domain expansion method. By constructing and training a neural convolutional network model, we utilize style loss and content loss functions, combined with peak signal-to-noise ratio evaluation, to generate high-quality domain-expanded images, thereby expanding the dataset and enhancing the model's generalization ability.
It improves the model's generalization ability, generates high-quality images, effectively expands the dataset, and enhances the accuracy and reliability of insulator detection.
Smart Images
Figure CN120876208A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision technology, and more specifically to a method and system for expanding image domains based on domain style. Background Technology
[0002] Increased electricity demand has increased the capacity of transmission lines, thereby raising the requirements for transmission reliability. Power inspections, crucial for maintaining grid stability, have become more complex and labor-intensive. Insulators, critical components of transmission lines, are constantly exposed to harsh outdoor conditions, increasing the risk of defects and failures. Timely inspections are essential to ensuring the safe operation of insulators, mitigating safety hazards, and improving the reliability of the power system. In recent years, unmanned aerial vehicles (UAVs) have increasingly replaced manual inspections, and computer vision technology has been widely applied to insulator monitoring. However, data shortages and domain translation between datasets significantly impact model performance and generalization ability.
[0003] Unlike typical computer vision tasks that usually have sufficient datasets, current power vision defect detection tasks face a severe data shortage. The reasons for this scarcity include: the lower frequency of defects compared to normal data; challenges in acquiring data in specific environments; and the confidentiality issues associated with data primarily held by power grid companies, with most datasets being private and characterized by insufficient defect images, limited insulator types, and structural constraints, leading to uneven data distribution and data domain shifts.
[0004] In power defect detection, domain transfer is often overlooked, as previous methods assumed that training and testing data came from the same domain. In reality, detection often involves unseen domains, and defect data is limited, making it challenging to train an acceptable detection model. Insufficient data quantity and detection scenarios are the main reasons for domain transfer. Most work focuses on increasing the amount of data rather than examining the specific scenarios. This includes using data augmentation techniques to generate augmented rather than new data; manually segmenting insulator instances and pasting them onto new background images (this requires manual intervention and cannot simulate new camera angles or scenarios); or generating images of self-exploding defects by modifying defect regions, but this is limited to specific defect types and unchanging camera angles.
[0005] Therefore, how to propose a domain-style-based image domain expansion method and system to effectively expand the dataset is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0006] In view of this, the present invention provides a method and system for image domain expansion based on domain style, which effectively expands the dataset and enhances the generalization ability of the model.
[0007] To achieve the above objectives, the present invention adopts the following technical solution:
[0008] On the one hand, this invention proposes an image domain expansion method based on domain style, comprising the following steps:
[0009] Get the content image and style image;
[0010] A domain transfer model based on a neural convolutional network is constructed and trained. The loss function of the domain transfer model includes style loss and content loss, wherein the content loss adds different weights to different instance parts of the content image.
[0011] The content image and the style image are input into the trained domain transformation model, and the domain expansion image is output.
[0012] Preferably, the different instance portions of the content image include: a background portion, a problem-free portion, and a defective portion.
[0013] Preferably, the loss function The formula is as follows:
[0014]
[0015] In the formula, For content loss, For style loss, α and β are the content loss weights and style loss weights, respectively.
[0016] Preferably, the content loss It is the sum of the mean squared errors (MSEs) of the l-th layer feature maps between the content image C and the output style transfer image T, as shown in the following formula:
[0017]
[0018] In the formula, ω0, ω1, and ω2 represent the weights of the background portion, the no-problem instance, and the defective instance, respectively; M k This is the binary mask image for the k-th part; This represents the pixel at position (i,j) of the content image corresponding to the k-th instance part; This represents the pixel at position (i,j) of the output style transfer image corresponding to the kth instance part.
[0019] Preferably, the style loss The formula is as follows:
[0020]
[0021] In the formula, L is the total number of layers in the neural convolutional network, S is the style image, and Gram(·) is a function to calculate the Gram matrix.
[0022] Preferably, a domain-style-based image domain expansion method further includes: calculating the image content quality of each instance part using peak signal-to-noise ratio to determine whether the output image is qualified; the output image is an image obtained by passing the content image through a neural convolutional network;
[0023] Different PSNR thresholds are set for each instance portion. When the peak signal-to-noise ratio of each instance portion is greater than the corresponding PSNR threshold, the output image is considered qualified.
[0024] Preferably, the peak signal-to-noise ratio (PSNR) is calculated using the following formula:
[0025]
[0026] In the formula, MAX is 255; m and n are the width and height of the output image, respectively; I(i,j) is the pixel value at position (i,j) of the output image, K(i,j) is the pixel value at position (i,j) of the input image; and M is the mask corresponding to each instance part.
[0027] On the other hand, the present invention also proposes a domain style-based image domain expansion system for implementing the aforementioned domain style-based image domain expansion method. The system includes:
[0028] The image acquisition module is used to acquire content images and style images;
[0029] The model building module is used to build and train a domain transfer model based on a neural convolutional network. The loss function of the domain transfer model includes style loss and content loss, and the content loss adds different weights to different instance parts of the content image.
[0030] The domain expansion module is used to input the content image and the style image into the trained domain transformation model and output the domain expansion image.
[0031] Preferably, a domain-style-based image domain expansion system further includes:
[0032] The image quality assessment module is used to calculate the image content quality of each instance using peak signal-to-noise ratio in order to determine whether the output image is qualified.
[0033] Different PSNR thresholds are set for each instance portion. When the peak signal-to-noise ratio of each instance portion is greater than the corresponding PSNR threshold, the output image is considered qualified.
[0034] As can be seen from the above technical solutions, compared with the prior art, this invention discloses a method and system for image domain expansion based on domain style. By constructing and training a neural convolutional network model, it achieves the conversion from content images to domain-expanded images. By introducing a weight allocation mechanism for content loss and using peak signal-to-noise ratio for quality assessment, this invention not only improves the quality of the converted images but also effectively expands the dataset and enhances the generalization ability of the model, showing broad application prospects in image processing, computer vision, and other fields. Attached Figure Description
[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0036] Figure 1 This is a schematic diagram of the method flow provided by the present invention;
[0037] Figure 2 Diagram illustrating the domain expansion;
[0038] Figure 3(a) is the original image, and Figure 3(b) is the domain-extended image obtained using the domain extension method of the present invention. Detailed Implementation
[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0040] On the one hand, this invention proposes an image domain extension method based on domain style, such as Figure 1 As shown, it includes the following steps:
[0041] S1. Obtain the content image and style image.
[0042] The content image in this embodiment is an image sample of an insulator disk.
[0043] In this embodiment, images from two categories ("artistic painting" and "photograph") are selected from the PACS dataset (a dataset widely used for domain generalization), and images from two categories ("smog" and "clear sky") are selected from the Internet as style images. The "artistic painting" category is an artificial domain image, while the other three categories are real-world images. Figure 2 This illustrates the process of extending the method proposed in this embodiment from one source domain to multiple domains.
[0044] S2. Construct and train a domain transfer model based on a neural convolutional network. The loss function of the domain transfer model includes style loss and content loss. The content loss assigns different weights to different instance parts of the content image. The different instance parts of the content image include: background, problem-free instances, and defective instances. (See reference...) Figure 2 The lower left portion. In this embodiment, different instance portions of the content image are obtained through instance segmentation.
[0045] In this embodiment, the domain transfer model based on neural convolutional networks is a general style transfer network. The network structure can be referenced in "Neural Transfer Using PyTorch—PyTorch Tutorials 2.5.0+cu124documentation" and optimized and trained using a loss function.
[0046] Specifically, the loss function The formula is as follows:
[0047]
[0048] In the formula, For content loss, For style loss, α and β are the content loss weights and style loss weights, respectively.
[0049] Content loss is defined as the sum of the mean squared errors (MSEs) of the l-th layer feature maps between the content image C and the output style transfer image T (i.e., the domain-extended image obtained through the domain transfer model). This embodiment adds different weights to the background, problem-free instances, and defective instances to control content quality and content loss. The formula is as follows:
[0050]
[0051] In the formula, ω0, ω1, and ω2 represent the weights of the background portion, the no-problem instance, and the defective instance, respectively, according to... Figure 2 As can be seen from the lower left corner, in this embodiment, ω0 = 1.0, ω1 = 1.5, and ω2 = 2.0; M k This is a binary mask image of the k-th part (k = 0, 1, 2, where k = 0 represents the background part, k = 1 represents an instance without problems, and k = 2 represents an instance with defects). A pixel value of 1 indicates a specific instance part, and 0 indicates other parts. This represents the pixel at position (i,j) of the content image corresponding to the k-th instance part; This represents the pixel at position (i,j) of the output style transfer image corresponding to the kth instance part.
[0052] Style loss The formula is as follows:
[0053]
[0054] In the formula, L is the total number of layers in the neural network, S is the style image, and Gram(·) is a function to calculate the Gram matrix, which is obtained by multiplying the input matrix by its transpose matrix and normalizing each element by dividing it by the total number of elements in the matrix.
[0055] S3. Input the content image and style image into the trained domain transformation model and output the domain expansion image. The original image, i.e. the content image, is shown in Figure 3(a), and the corresponding domain expansion image is shown in Figure 3(b).
[0056] Furthermore, a domain-style-based image domain expansion method also includes: calculating the image content quality of each instance part using peak signal-to-noise ratio to determine whether the output image is qualified; the output image is the image obtained by passing the content image through a neural convolutional network;
[0057] Different PSNR thresholds are set for each instance portion. When the peak signal-to-noise ratio of each instance portion is greater than the corresponding PSNR threshold, the output image is considered qualified.
[0058] The formula for calculating Peak Signal-to-Noise Ratio (PSNR) is as follows:
[0059]
[0060] In the formula, MAX is 255; m and n are the width and height of the output image, respectively; I(i,k) is the pixel value at position (i,j) of the output image, K(i,j) is the pixel value at position (i,j) of the input image; and M is the mask corresponding to each instance part.
[0061] In this embodiment, different PSNR thresholds are applied to the foreground and background of each sample to prevent the model from destroying the content information of the defective insulator disk. Since the content information is crucial to defective instances, a higher threshold is applied to it. Disks marked as problem-free are assigned a higher threshold than the background. Only when the PSNR value of each instance portion is greater than the threshold is the image quality considered acceptable and saved. After transferring each sample to four other domains, the result is obtained from domain D. S0 To Domain D S4 Data (reference) Figure 2 (Images of right-hand side, domains 0-4).
[0062] On the other hand, the present invention also proposes a domain style-based image domain expansion system for implementing the aforementioned domain style-based image domain expansion method. The system includes:
[0063] The image acquisition module is used to acquire content images and style images;
[0064] The model building module is used to build and train a domain transfer model based on a neural convolutional network. The loss function of the domain transfer model includes style loss and content loss. The content loss adds different weights to different instance parts of the content image.
[0065] The domain extension module is used to input content images and style images into the trained domain transformation model and output a domain extension image.
[0066] Preferably, a domain-style-based image domain expansion system further includes:
[0067] The image quality assessment module is used to calculate the image content quality of each instance using peak signal-to-noise ratio in order to determine whether the output image is qualified.
[0068] Different PSNR thresholds are set for each instance portion. When the peak signal-to-noise ratio of each instance portion is greater than the corresponding PSNR threshold, the output image is considered qualified.
[0069] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.
[0070] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for expanding image domains based on domain style, characterized in that, Includes the following steps: Get the content image and style image; A domain transfer model based on a neural convolutional network is constructed and trained. The loss function of the domain transfer model includes style loss and content loss, wherein the content loss adds different weights to different instance parts of the content image. The content image and the style image are input into the trained domain transformation model, and the domain expansion image is output.
2. The image domain expansion method based on domain style according to claim 1, characterized in that, The different instances of the content image include: background, instances without problems, and instances with defects.
3. The image domain expansion method based on domain style according to claim 2, characterized in that, The loss function The formula is as follows: In the formula, For content loss, For style loss, α and β are the content loss weights and style loss weights, respectively.
4. The image domain expansion method based on domain style according to claim 3, characterized in that, The content loss It is the sum of the mean squared errors (MSEs) of the l-th layer feature maps between the content image C and the output style transfer image T, as shown in the following formula: In the formula, ω0, ω1, and ω2 represent the weights of the background portion, the no-problem instance, and the defective instance, respectively; M k This is the binary mask image for the k-th part; This represents the pixel at position (i,j) of the content image corresponding to the k-th instance part; This represents the pixel at position (i,j) of the output style transfer image corresponding to the kth instance part.
5. The image domain expansion method based on domain style according to claim 4, characterized in that, The style loss The formula is as follows: In the formula, L is the total number of layers in the neural convolutional network, S is the style image, and Gram(·) is a function to calculate the Gram matrix.
6. The image domain expansion method based on domain style according to claim 2, characterized in that, Also includes: The peak signal-to-noise ratio is used to calculate the image content quality of each instance portion in order to determine whether the output image is qualified; the output image is the image obtained by passing the content image through a neural convolutional network. Different PSNR thresholds are set for each instance portion. When the peak signal-to-noise ratio of each instance portion is greater than the corresponding PSNR threshold, the output image is considered qualified.
7. The image domain expansion method based on domain style according to claim 6, characterized in that, The formula for calculating the peak signal-to-noise ratio (PSNR) is as follows: In the formula, MAX is 255; m and n are the width and height of the output image, respectively; I(i,k) is the pixel value at position (i,j) of the output image, K(i,j) is the pixel value at position (i,j) of the input image; and M is the mask corresponding to each instance part.
8. A domain-style-based image domain extension system, characterized in that, include: The image acquisition module is used to acquire content images and style images; The model building module is used to build and train a domain transfer model based on a neural convolutional network. The loss function of the domain transfer model includes style loss and content loss, and the content loss adds different weights to different instance parts of the content image. The domain expansion module is used to input the content image and the style image into the trained domain transformation model and output the domain expansion image.
9. A domain-style-based image domain expansion system according to claim 8, characterized in that, Also includes: The image quality assessment module is used to calculate the image content quality of each instance using peak signal-to-noise ratio in order to determine whether the output image is qualified. Different PSNR thresholds are set for each instance portion. When the peak signal-to-noise ratio of each instance portion is greater than the corresponding PSNR threshold, the output image is considered qualified.