Roof photovoltaic sample generation and segmentation method based on generative artificial intelligence

Abstract

The invention discloses a roof photovoltaic sample generation and segmentation method based on generative artificial intelligence, relates to the technical field of artificial intelligence, and is used for solving the problems of lack of high-quality samples, poor model generalization ability and low segmentation precision in the prior art. The roof photovoltaic sample generation and segmentation method based on generative artificial intelligence comprises the steps of obtaining a roof photovoltaic image and a segmentation mask corresponding to the photovoltaic image, and designing text description of the photovoltaic image; configuring a text-guided stable diffusion repair model, and training the stable diffusion repair model; designing a text prompt for generating a target image, and generating a plurality of target images of the same photovoltaic image; mixing the photovoltaic image with a target image corresponding to the photovoltaic image, and pairing the target image, the photovoltaic image and the segmentation mask to form a photovoltaic segmentation sample set; and determining a segmentation algorithm, training the segmentation algorithm by using the segmentation sample set, and segmenting the roof photovoltaic image by using the segmentation algorithm.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more specifically, to a method for generating and segmenting roof photovoltaic samples based on generative artificial intelligence. Background Art

[0002] Traditional methods for assessing solar potential and installation details mainly rely on on-site surveys and reports, and mainly focus on the residential scale. Due to insufficient accuracy, it is difficult to meet the needs of large-scale urban planning and management. In recent years, with the development of deep learning technology, more and more research has begun to use aerial images and remote sensing technology to identify roof photovoltaic facilities, so as to be able to report residential photovoltaic facilities more accurately.

[0003] Existing deep learning technologies can be widely used to identify roof photovoltaic facilities, but their identification efficiency is affected by sensitivity to different and heterogeneous environments. Existing identification tools usually require a large amount of labeled data for training, which not only increases the cost of data collection and annotation, but also limits the generalization ability of the model. Moreover, when dealing with complex backgrounds and environments, existing technologies are difficult to accurately identify the edge details of photovoltaic panels, resulting in insufficient segmentation accuracy. It can be seen that the existing technology has technical problems such as lack of high-quality samples, poor model generalization ability, and low segmentation accuracy. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for generating and segmenting roof photovoltaic samples based on generative artificial intelligence, which is used to solve the problems of lack of high-quality samples, poor model generalization ability, and low segmentation accuracy existing in the prior art. In view of this, the present invention is realized through the following solutions.

[0005] The present invention provides a method for generating and segmenting roof photovoltaic samples based on generative artificial intelligence, including: Obtain roof photovoltaic images, as well as the segmentation masks corresponding to the photovoltaic images, and design text descriptions of the photovoltaic images; Configure a text-guided stable diffusion inpainting model, and use the text description, photovoltaic image, and the segmentation mask corresponding to the photovoltaic image to train the stable diffusion inpainting model; Design text prompts for generating target images, and input the text prompts, photovoltaic images, and the segmentation masks corresponding to the photovoltaic images into the trained stable diffusion inpainting model to generate target images; repeat this process to generate multiple target images of the same photovoltaic image; Generate target images for each photovoltaic image; mix the photovoltaic images and the corresponding target images, and pair the target images, photovoltaic images, and segmentation masks to form a photovoltaic segmentation sample set; Determine the segmentation algorithm. After training the segmentation algorithm using the segmentation sample set, use the segmentation algorithm to segment the rooftop photovoltaic image.

[0006] Compared with the prior art, in the method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence of the present invention, after obtaining the rooftop photovoltaic image and the corresponding segmentation mask, design the text description of the photovoltaic image; after configuring the text-guided Stable Diffusion inpainting model, use the text description, the photovoltaic image, and the segmentation mask to train the Stable Diffusion inpainting model; further, design the text prompt for generating the target image, and input the text prompt, the photovoltaic image, and the segmentation mask corresponding to the photovoltaic image into the trained Stable Diffusion inpainting model to generate the target image, and repeat this process to generate multiple target images (i.e., diverse target images) of the same photovoltaic image; further, this solution generates target images for each photovoltaic image; mix the photovoltaic image and the corresponding target image, and pair the target image, the photovoltaic image, and the segmentation mask to form a photovoltaic segmentation sample set; after obtaining the segmentation sample set, determine the segmentation algorithm, and use the segmentation sample set to train the segmentation algorithm, and then use the segmentation algorithm to segment the rooftop photovoltaic image; through the above technical solution, on the one hand, the generation of rooftop photovoltaic samples is realized, and on the other hand, the segmentation of rooftop photovoltaic images is realized. In the above solution of the present invention, diverse rooftop photovoltaic data (i.e., the above-mentioned segmentation sample set) is generated through generative artificial intelligence technology, reducing the dependence on real data, and combining the generated data and a small amount of real data to optimize the segmentation algorithm, improving the recognition ability of the edge details of photovoltaic panels; in this solution, after generating diverse backgrounds and environments, the generalization ability of the model can be enhanced. Through the above technical solution of the present invention, the problems existing in the prior art, such as the lack of high-quality samples, poor generalization ability of the model, and low segmentation accuracy, are solved.

[0007] Further, in the method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence of the present invention, the text-guided Stable Diffusion inpainting model is the Stable Diffusion 2 base model; the configuration of the text-guided Stable Diffusion inpainting model includes: Load the weights of the pre-trained Stable Diffusion 2 model. Set the initial learning rate of the Stable Diffusion 2 model to 0.0001, and warm up in the first 10,000 steps, and the maximum model training expansion is 200,000 steps. Set the optimizer for training the Stable Diffusion 2 model to the Adam optimizer, and the loss function to the mean squared error loss function.

[0008] Further, in the method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence of the present invention, the first exponent of the squared computational gradient of the Adam optimizer is 0.9, and the second exponent is 0.999.

[0009] Further, in the method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence of the present invention, training the StableDiffusion inpainting model includes: Confirming that the size of the input photovoltaic image is consistent with the segmentation mask; Converting the photovoltaic image into a latent representation z = E(x) through the encoder E; where z is the latent representation, E is the encoder, and x is the input photovoltaic image; Adding Gaussian noise ϵ to the latent representation z and generating a noisy image z K , and the process of adding Gaussian noise follows a fixed Markov chain to increase the noise intensity; Performing a reverse diffusion process by the StableDiffusion inpainting model to remove the noise and restore the photovoltaic image; Among them, the StableDiffusion inpainting model predicts the noise residual, subtracts the noise residual from the latent representation z, and reconstructs the denoised latent representation into the final image x̃ = D(z K ), where x̃ represents the restored photovoltaic image, D(z K ) represents the reverse diffusion process, and z K represents the noisy image.

[0010] Further, in the method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence of the present invention, generating multiple target images of the same photovoltaic image includes: Generating target images with various backgrounds and environments by changing the text prompt; Adjusting the shape and position of the segmentation mask to generate target images with different occlusions of the photovoltaic image; Performing quality assessment on the target images and removing the blurred and distorted target images; Performing rotation, flipping, and cropping operations on the remaining target images to increase diversity.

[0011] Further, in the method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence of the present invention, during the process of mixing the photovoltaic image and the corresponding target image, the mixing ratio of the target image to the photovoltaic image is 10:1.

[0012] Further, in the method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence of the present invention, after mixing the photovoltaic image and the corresponding target image, it further includes: Labeling the target image.

[0013] Furthermore, in the method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence of the present invention, the segmentation algorithm includes a deep learning algorithm, and the training of the segmentation algorithm using the segmentation sample set includes: Dividing the segmentation sample set into a training set, a validation set, and a test set; the ratio of the training set, the validation set, and the test set in the segmentation sample set is 7:2:1; Training the segmentation algorithm using the segmentation sample set; during the training process, evaluating the model performance on the validation set, adjusting the model parameters according to the validation results, and optimizing the model performance.

[0014] Furthermore, in the method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence of the present invention, after training the segmentation algorithm using the segmentation sample set, it further includes: Evaluating the final performance of the segmentation algorithm on the test set, and recording the segmentation accuracy IoU, precision Precision, and recall Recall of the segmentation algorithm.

[0015] Furthermore, in the method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence of the present invention, the target image includes the same rooftop photovoltaic as the input photovoltaic image, as well as the background environment depicted by the text prompt. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present invention, and constitute a part of the present invention. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention, and do not constitute an improper limitation to the present invention. In the drawings: Figure 1 It is a schematic diagram of the process of generating new samples by the text-guided stable diffusion repair model of the present invention; Figure 2 It is a schematic diagram of generating diverse segmentation samples of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] In order to make the technical problems, technical solutions, and beneficial effects to be solved by the present invention clearer, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0018] It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element.

[0019] In addition, the terms "first" and "second" are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality of" means two or more, unless otherwise specifically defined. "Several" means one or more, unless otherwise specifically defined.

[0020] Existing deep learning techniques can be widely used to identify rooftop photovoltaic facilities, but their recognition efficiency is greatly affected by the sensitivity to different and heterogeneous environments. Existing recognition tools usually require a large amount of labeled data for training, which not only increases the cost of data collection and annotation, but also limits the generalization ability of the model. Moreover, when dealing with complex backgrounds and environments, existing technologies are difficult to accurately identify the edge details of photovoltaic panels, resulting in insufficient segmentation accuracy. It can be seen that the existing technologies have technical problems such as a lack of high-quality samples, poor model generalization ability, and low segmentation accuracy.

[0021] To solve the above technical problems, the present invention provides a method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence, including: Obtain a rooftop photovoltaic image, as well as the segmentation mask corresponding to the photovoltaic image, and design a text description of the photovoltaic image; Configure a text-guided stable diffusion inpainting model, and use the text description, the photovoltaic image, and the segmentation mask corresponding to the photovoltaic image to train the stable diffusion inpainting model; Design a text prompt for generating a target image, and input the text prompt, the photovoltaic image, and the segmentation mask corresponding to the photovoltaic image into the trained stable diffusion inpainting model to generate a target image; repeat this process to generate multiple target images of the same photovoltaic image; Generate a target image for each photovoltaic image; mix the photovoltaic image and the corresponding target image, and pair the target image, the photovoltaic image, and the segmentation mask to form a photovoltaic segmentation sample set; Determine a segmentation algorithm, use the segmentation sample set to train the segmentation algorithm, and then use the segmentation algorithm to segment the rooftop photovoltaic image.

[0022] In the case of adopting the above technical solution, in the method for generating and segmenting roof photovoltaic samples based on generative artificial intelligence of the present invention, after obtaining the roof photovoltaic image and the corresponding segmentation mask, a text description of the photovoltaic image is designed; after configuring a text-guided stable diffusion inpainting model, the stable diffusion inpainting model is trained using the text description, the photovoltaic image, and the segmentation mask; further, a text prompt for generating the target image is designed, and the text prompt, the photovoltaic image, and the segmentation mask corresponding to the photovoltaic image are input into the trained stable diffusion inpainting model to generate the target image, and this process is repeated to generate multiple target images (i.e., diverse target images) of the same photovoltaic image; further, this solution generates target images for each photovoltaic image; the photovoltaic image and the corresponding target image are mixed, and the target image, the photovoltaic image, and the segmentation mask are paired to form a photovoltaic segmentation sample set; after obtaining the segmentation sample set, a segmentation algorithm is determined, and the segmentation algorithm is trained using the segmentation sample set, and then the segmentation algorithm is used to segment the roof photovoltaic image; through the above technical solution, on the one hand, the generation of roof photovoltaic samples is realized, and on the other hand, the segmentation of roof photovoltaic images is realized. In the above solution of the present invention, diverse roof photovoltaic data (i.e., the above segmentation sample set) is generated through generative artificial intelligence technology, reducing the dependence on real data, and combining the generated data and a small amount of real data to optimize the segmentation algorithm, improving the recognition ability of the edge details of photovoltaic panels; in this solution, after generating diverse backgrounds and environments, the generalization ability of the model can be enhanced. Through the above technical solution of the present invention, the problems of the existing technology such as the lack of high-quality samples, poor generalization ability of the model, and low segmentation accuracy are solved.

[0023] To better understand the present invention, the content of the present invention will be further clarified below in conjunction with specific embodiments, but the content of the present invention is not limited to the following embodiments. Embodiment

[0024] This embodiment provides a method for generating and segmenting roof photovoltaic samples based on generative artificial intelligence, including: Step 1, obtain a roof photovoltaic image and the segmentation mask corresponding to the photovoltaic image, and design a text description of the photovoltaic image; Step 2, configure a text-guided stable diffusion inpainting model, and train the stable diffusion inpainting model using the text description, the photovoltaic image, and the segmentation mask corresponding to the photovoltaic image; Step 3, design a text prompt for generating a target image, and input the text prompt, the photovoltaic image, and the segmentation mask corresponding to the photovoltaic image into the trained stable diffusion inpainting model to generate a target image; repeat this process to generate multiple target images of the same photovoltaic image; Step 4, generate a target image for each photovoltaic image; mix the photovoltaic image and its corresponding target image, and pair the target image, photovoltaic image, and segmentation mask to form a photovoltaic segmentation sample set; Step 5, determine the segmentation algorithm, and after training the segmentation algorithm using the segmentation sample set, use the segmentation algorithm to segment the rooftop photovoltaic image. Embodiment

[0025] This embodiment provides a method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence, including: S100, obtain a rooftop photovoltaic image, the corresponding segmentation mask of the photovoltaic image, and design a text description of the photovoltaic image; S200, configure a text-guided Stable Diffusion inpainting model, and train the Stable Diffusion inpainting model using the text description, photovoltaic image, and the corresponding segmentation mask of the photovoltaic image; the text-guided Stable Diffusion inpainting model is a Stable Diffusion 2 base model, and the configuration of the text-guided Stable Diffusion inpainting model includes: S211, load the weights of the pre-trained Stable Diffusion 2 model; S212, set the initial learning rate of the Stable Diffusion 2 model to 0.0001, and gradually warm up in the first 10,000 steps, with the maximum model training extended to 200,000 steps; S213, set the optimizer for training the Stable Diffusion 2 model to the Adam optimizer, and the loss function to the mean squared error loss function; the first exponent of the Adam optimizer for calculating the gradient machine square is 0.9, and the second exponent is 0.999; Further, the training of the Stable Diffusion inpainting model includes: S221, confirm that the size of the input photovoltaic image is consistent with the segmentation mask; S222, convert the photovoltaic image into a latent representation z = E(x) through the encoder E; where z is the latent representation, E is the encoder, and x is the input photovoltaic image; S223, add Gaussian noise ϵ to the latent representation z and generate a noisy image z K , and the process of adding Gaussian noise follows a fixed Markov chain to gradually increase the noise intensity; S224, the Stable Diffusion inpainting model performs a reverse diffusion process to gradually eliminate the noise and restore the photovoltaic image; where the Stable Diffusion inpainting model predicts the noise residual, subtracts the noise residual from the latent representation z, and reconstructs the denoised latent representation into the final image x̃ = D(z K), where ỹ represents the restored photovoltaic image, and D(z K ) represents the reverse diffusion process, and z K represents the noisy image; S300. Design a text prompt for generating a target image. Input the text prompt, the photovoltaic image, and the segmentation mask corresponding to the photovoltaic image into the trained stable diffusion inpainting model to generate a target image; repeat this process to generate multiple target images of the same photovoltaic image; the target image includes the rooftop photovoltaic that is the same as the input photovoltaic image and the background environment depicted by the text prompt; Furthermore, generating multiple target images of the same photovoltaic image includes: S301. Generate target images with various backgrounds and environments by changing the text prompt; S302. Adjust the shape and position of the segmentation mask to generate target images with different occlusions of the photovoltaic image; S303. Perform quality assessment on the target images and remove the blurred and distorted target images; S304. Perform rotation, flipping, and cropping operations on the remaining target images to increase diversity; S400. Generate a target image for each photovoltaic image; mix the photovoltaic image and the corresponding target image, and pair the target image, the photovoltaic image, and the segmentation mask to form a photovoltaic segmentation sample set, and label the target image; wherein, the mixing ratio of the target image to the photovoltaic image is 10:1; in this embodiment, 2000 target images are generated and 200 photovoltaic images are mixed; S500. Determine a segmentation algorithm. After training the segmentation algorithm using the segmentation sample set, use the segmentation algorithm to segment the rooftop photovoltaic image; evaluate the final performance of the segmentation algorithm on the test set, and record the segmentation accuracy IoU, precision Precision, and recall Recall of the segmentation algorithm; wherein, the segmentation algorithm includes a deep learning algorithm; the segmentation algorithm is U-Net; in this embodiment, the U-Net segmentation model is trained using the segmentation sample set, and the number of training iterations is 160,000 times; segmentation result: the segmentation accuracy (IoU) reaches 70.69%; Furthermore, the training of the segmentation algorithm using the segmentation sample set includes: S501. Divide the segmentation sample set into a training set, a validation set, and a test set; the ratio of the training set, the validation set, and the test set in the segmentation sample set is 7:2:1; S502. Use the segmentation sample set to train the segmentation algorithm; evaluate the model performance on the validation set during the training process, adjust the model parameters according to the validation results, and optimize the model performance. Embodiment

[0026] Please refer to Figure 1 and Figure 2 , this embodiment provides a method for generating and segmenting roof photovoltaic samples based on generative artificial intelligence, including: S100, collect high-quality real roof photovoltaic images and the corresponding segmentation masks from public datasets or on-site shootings, preprocess the collected photovoltaic images, including cropping, scaling, and normalization, to ensure consistent image sizes for subsequent processing, and design text prompts (i.e., text descriptions) for the photovoltaic images; the text prompt can be "a very detailed aerial photo showing the photovoltaic panels on the roof", and high-quality images can be generated according to the text prompt; in practice, the text prompt should be as detailed as possible to ensure that the generated images conform to the actual scenario; S200, configure a text-guided Stable Diffusion inpainting model, and train the Stable Diffusion inpainting model using the text prompt, the photovoltaic image, and the corresponding segmentation mask of the photovoltaic image; the text-guided Stable Diffusion inpainting model is the Stable Diffusion 2 base model, which can generate high-quality images according to text prompts and performs well in repairing the masked areas of images; during the training of the Stable Diffusion inpainting model, regularly evaluate the model performance on the validation set and adjust the model parameters according to the validation results to optimize the quality of the generated images; the configuration of the text-guided Stable Diffusion inpainting model includes: S211, load the weights of the pre-trained Stable Diffusion 2 model; S212, set the initial learning rate of the Stable Diffusion 2 model to 0.0001 and gradually warm up within the first 10,000 steps, with the maximum model training extension being 200,000 steps; S213, set the optimizer for training the Stable Diffusion 2 model to the Adam optimizer and the loss function to the mean squared error loss function; the first exponent of the Adam optimizer for calculating the gradient machine square is 0.9, and the second exponent is 0.999; Furthermore, the training of the Stable Diffusion inpainting model includes: S221, confirm that the size of the input photovoltaic image is consistent with the segmentation mask; S222, convert the photovoltaic image into a latent representation z = E(x) through the encoder E; where z is the latent representation, E is the encoder, and x is the input photovoltaic image; S223, add Gaussian noise ϵ to the latent representation z and generate a noisy image z K , and the process of adding Gaussian noise follows a fixed Markov chain to gradually increase the noise intensity; S224. The stable diffusion inpainting model performs a reverse diffusion process to gradually eliminate noise and restore the photovoltaic image. Among them, the stable diffusion inpainting model predicts the noise residual, subtracts the noise residual from the latent representation z, and reconstructs the denoised latent representation into the final image x̃ = D(z K ), where x̃ represents the restored photovoltaic image, and D(z K ) represents the reverse diffusion process, and z K represents the noise image; S300. Design a text prompt for generating the target image, and input the text prompt, the photovoltaic image, and the segmentation mask corresponding to the photovoltaic image into the trained stable diffusion inpainting model to generate the target image. Repeat this process to generate multiple target images of the same photovoltaic image. The target image includes the rooftop photovoltaic that is the same as the input photovoltaic image and the background environment depicted by the text prompt; Furthermore, the generated target image can be optimized through multiple iterations to ensure that the generated target image is similar to the real image in terms of background and details, while retaining the key photovoltaic panel features. Perform a quality assessment on the generated target image to ensure that the clarity and details of the image meet the requirements; Furthermore, generating multiple target images of the same photovoltaic image includes: S301. By changing the text prompt, such as "busy city background", "suburban environment", etc., generate target images with various backgrounds and environments, that is, generate diverse backgrounds and environments; S302. Adjust the shape and position of the segmentation mask to generate target images with different occlusions of the photovoltaic image to further enhance the diversity; S303. Perform a quality assessment on the target image and remove the blurred and distorted target images; S304. Perform rotation, flipping, and cropping operations on the remaining target images to increase the diversity; S400. Generate a target image for each photovoltaic image. Mix the photovoltaic image and the corresponding target image, and pair the target image, the photovoltaic image, and the segmentation mask to form a photovoltaic segmentation sample set. And annotate the target image. During the annotation process, use a professional image annotation tool (such as LabelImg) to annotate the generated image to ensure the accuracy and consistency of the annotation. Mark the position and boundary of the photovoltaic panel to generate a segmentation mask. Among them, the mixing ratio of the target image to the photovoltaic image is 10:1. In this embodiment, 2000 target images are generated and 200 photovoltaic images are mixed; S500. Determine the deep learning algorithm as the segmentation algorithm. The deep learning algorithm can be U-Net or Mask R-CNN, and configure the model parameters according to specific tasks, including the learning rate, optimizer, loss function, etc. After training the segmentation algorithm with the segmentation sample set, use the segmentation algorithm to segment the rooftop photovoltaic images. Evaluate the final performance of the segmentation algorithm on the test set, and record the segmentation accuracy IoU, precision Precision, and recall Recall of the segmentation algorithm. Among them, the segmentation algorithm includes a deep learning algorithm; the segmentation algorithm is U-Net. In this embodiment, the U-Net segmentation model is trained using the segmentation sample set, and the number of training iterations is 160,000 times. Segmentation result: The segmentation accuracy (IoU) reaches 75.30%. Further, the training of the segmentation algorithm using the segmentation sample set includes: S501. Divide the segmentation sample set into a training set, a validation set, and a test set. The ratio of the training set, validation set, and test set in the segmentation sample set is 7:2:1. S502. Use the segmentation sample set to train the segmentation algorithm. During the training process, evaluate the model performance on the validation set, and adjust the model parameters according to the validation results to optimize the model performance.

[0027] Further, the model performance can be further optimized by adjusting hyperparameters such as the learning rate and batch size, and regularization techniques (such as Dropout and L2 regularization) are applied to prevent the model from overfitting. During the training process, data augmentation techniques (such as rotation, flipping, and cropping) are applied to increase data diversity and improve the generalization ability of the model.

[0028] Please refer to Figure 1 and Figure 2 , it can be seen that the present invention can obtain new samples by acquiring the label information in the original image (photovoltaic image), through text prompts, and the text-guided stable diffusion model. The new sample is the above-mentioned target image. Further, target images with various backgrounds and environments are generated by changing the text prompts. For example, as can be seen from Figure 2 , by changing the text prompt to "urban aerial image", sample 1 is generated, by changing the text prompt to "urban aerial image + there is a parking lot around", sample 2 is generated, and by changing the text prompt to "urban aerial image + sunny day", sample 3 is generated.

[0029] In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0030] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims described above.

Claims

1. A rooftop photovoltaic sample generation and segmentation method based on generative artificial intelligence, characterized in that: include: Acquire a rooftop photovoltaic image and a segmentation mask corresponding to the photovoltaic image, and design a text description of the photovoltaic image; configuring a text-guided stable diffusion inpainting model, and training the stable diffusion inpainting model using the text description, the photovoltaic image, and a segmentation mask corresponding to the photovoltaic image; Designing a text prompt for generating a target image, inputting the text prompt, the photovoltaic image, and the segmentation mask corresponding to the photovoltaic image into the trained stable diffusion inpainting model to generate the target image; repeating the process to generate multiple target images of the same photovoltaic image; Generating a target image for each photovoltaic image; mixing the photovoltaic image with the target image corresponding thereto, and pairing the target image, the photovoltaic image and the segmentation mask to form a photovoltaic segmentation sample set; A segmentation algorithm is determined, and after the segmentation algorithm is trained using the segmentation sample set, the segmentation algorithm is used to segment the roof photovoltaic image.

2. The method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence according to claim 1 is characterized in that: The text-guided stable diffusion repair model is a Stable Diffusion 2 base model; the configuration text-guided stable diffusion repair model includes: Load the weights of the pre-trained Stable Diffusion 2 model; The Stable Diffusion 2 model was initialized with a learning rate of 0.0001 and warmed up within the first 10,000 steps, with a maximum model training extension of 200,000 steps. Set the optimizer for training the Stable Diffusion 2 model to the Adam optimizer and the loss function to the mean square error loss function.

3. The method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence according to claim 2 is characterized in that: The first exponent of the calculated gradient machine square of the Adam optimizer is 0.9 and the second exponent is 0.

999.

4. The method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence according to claim 3 is characterized in that: The training of the stable diffusion repair model comprises: Confirm that the size of the input photovoltaic image is consistent with the segmentation mask; The photovoltaic image is converted into a potential representation z=E(x) through an encoder E; wherein z is the potential representation, E is the encoder, and x is the input photovoltaic image; Add Gaussian noise ϵ to the latent representation z and generate a noisy image z K ,The process of adding Gaussian noise follows a fixed Markov chain to increase the noise intensity; The stable diffusion repair model performs a reverse diffusion process to eliminate noise and restore the photovoltaic image; The stable diffusion inpainting model predicts the noise residual, subtracts it from the latent representation z, and reconstructs the denoised latent representation into the final image x̃ = D(z K ), x̃ represents the restored photovoltaic image, D(z K ) represents the reverse diffusion process, z K Represents a noisy image.

5. The method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence according to claim 4 is characterized in that: The method of generating a plurality of target images of the same photovoltaic image comprises: Generate target images with various backgrounds and environments by changing text prompts; Adjust the shape and position of the segmentation mask to generate target images with different masking of the photovoltaic image; Performing quality assessment on the target image and removing blur and distortion from the target image; The remaining target images are rotated, flipped, and cropped to increase diversity.

6. The method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence according to claim 5 is characterized in that: In the process of mixing the photovoltaic image with the corresponding target image, the mixing ratio of the target image to the photovoltaic image is 10:

1.

7. The method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence according to claim 6 is characterized in that: After mixing the photovoltaic image with the target image corresponding thereto, the method further comprises: The target image is labeled.

8. The method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence according to claim 7 is characterized in that: The segmentation algorithm includes a deep learning algorithm, and the step of training the segmentation algorithm using the segmentation sample set includes: The segmented sample set is divided into a training set, a validation set and a test set; the ratio of the training set, the validation set and the test set in the segmented sample set is 7:2:1; The segmentation algorithm is trained using the segmentation sample set; during the training process, the model performance is evaluated on the validation set, and the model parameters are adjusted according to the validation results to optimize the model performance.

9. The method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence according to claim 8, characterized in that: After the segmentation algorithm is trained using the segmentation sample set, the method further includes: The final performance of the segmentation algorithm is evaluated on the test set, and the segmentation accuracy IoU, precision Precision and recall Recall of the segmentation algorithm are recorded.

10. The method for generating and segmenting rooftop photovoltaic samples based on generative artificial intelligence according to claim 9, characterized in that: The target image includes the same rooftop photovoltaics as the input photovoltaic image, and the background environment depicted by the text prompt.

Images