Method and system for extracting ground objects from remote sensing images based on artificial intelligence diffusion model

Abstract

The present application provides a method and system for extracting objects from remote sensing images based on an artificial intelligence diffusion model, and relates to the field of image processing. The method includes constructing a training sample data set containing remote sensing images of uniform size and corresponding extracted true value masks, which can provide complete remote sensing images and expected output information for iterative training of improved diffusion models. The improved diffusion model includes two sub-networks UNet that independently process noise prediction and mask extraction, and accelerates convergence speed through sub-task decomposition. During the iterative training process, the Gaussian noise level is controlled according to the timestamp correlation coefficient and a noisy target image is generated with the true value mask, which can effectively enhance the robustness of the model in handling complex noise. The improved diffusion model not only improves the quality of object image extraction by decomposing the model construction of the task system and iterative training of noisy images, but also shortens the generation time of mask images in remote sensing image object extraction based on the diffusion model, thereby improving the object extraction speed.

Description

Technical Field

[0001] The present application relates to the field of image processing, and specifically to a method and system for extracting ground objects from remote sensing images based on an artificial intelligence diffusion model. Background Art

[0002] In the field of remote sensing, object extraction plays a very important role, such as urban planning, change detection, mapping, real-time monitoring, etc. Traditional methods encounter many difficulties and problems in object extraction, and recently deep learning methods have shown great potential and very good results in object extraction. Among many deep learning models, object extraction from remote sensing images is mainly based on semantic segmentation networks. The diffusion model has good noise robustness and has shown good results in the field of image generation. However, the application of diffusion models in the segmentation task of object extraction is still in its infancy. Since the diffusion model introduces Gaussian distribution parameters in each iteration during the generation of the object extraction segmentation task, and the number of iterations is large, it takes a long time to generate mask images through the diffusion model in object extraction. Summary of the invention

[0003] The present application provides a method and system for extracting ground objects from remote sensing images based on an artificial intelligence diffusion model, which is used to shorten the generation time of mask images in extracting ground objects from remote sensing images based on a diffusion model, thereby improving the speed of ground object extraction.

[0004] In a first aspect, the present application provides a method for extracting ground objects from remote sensing images based on an artificial intelligence diffusion model, the method comprising:

[0005] Construct a training sample data set, which includes remote sensing images of uniform size and extracted true value masks corresponding to the remote sensing images;

[0006] Constructing an improved diffusion model, the improved diffusion model includes a first UNet network and a second UNet network, wherein the first UNet network is used to predict noise, and the second UNet network is used to predict a mask image;

[0007] When training the improved diffusion model through the training sample data set, the remote sensing image of uniform size is used as the conditional image x0, and the relevant timestamp coefficient γ generated based on the preset number of iteration steps t and the coefficient γ based on the relevant timestamp t , randomly generated standard Gaussian noise ε and extracted true value mask y0 to generate a noisy target image y t As input, the improved diffusion model is iteratively trained, and when the accuracy of the improved diffusion model reaches a preset standard, the target improved diffusion model is obtained;

[0008] Obtain the remote sensing image to be processed, and generate a standard normal distribution array based on the remote sensing image to be processed and a preset number of iteration steps;

[0009] The remote sensing image to be processed and the standard normal distribution array are split into a number of image blocks and the standard normal distribution arrays at corresponding positions, and the number of image blocks and the standard normal distribution arrays at corresponding positions are input into the target improved diffusion model to obtain mask images corresponding to the number of image blocks;

[0010] The mask images corresponding to several image blocks are spliced ​​to obtain a target mask image.

[0011] By adopting the above technical solution, a training sample data set containing remote sensing images of uniform size and corresponding extracted true value masks is constructed, which can provide complete remote sensing images and expected output information for iterative training of improved diffusion models. The improved diffusion model contains two sub-networks UNet that independently process noise prediction and mask extraction. The difficulty of model training is reduced and the convergence speed is accelerated by subtask decomposition. During the iterative training process, the Gaussian noise level is controlled according to the timestamp correlation coefficient and a noisy target image is generated with the true value mask, which can effectively enhance the robustness of the model in processing complex noise. After obtaining the trained improved diffusion model, the mask output can be quickly generated for the split remote sensing image blocks, and then the complete target mask image reflecting the overall distribution of the ground objects can be obtained through splicing processing. In this way, the improved diffusion model not only improves the quality of ground object image extraction by decomposing the model construction of the task system and iterative training of noisy images, but also greatly shortens the generation time of mask images in remote sensing image ground object extraction based on the diffusion model, so that the ground object extraction speed is improved.

[0012] Optionally, when training the improved diffusion model through the training sample data set, the remote sensing image of uniform size is used as the conditional image x0, and the relevant timestamp coefficient γ generated based on the preset number of iteration steps t and the coefficient γ based on the relevant timestamp t , randomly generated standard Gaussian noise ε and extracted true value mask y0 to generate a noisy target image y t As input, the improved diffusion model is iteratively trained, including:

[0013] According to the preset number of iterations T, set the number of hyperparameters α to T t ;

[0014] A timestamp t is randomly selected in the interval 0 to T, and the t and timestamp t to generate the related timestamp coefficient γ t ;

[0015] In each iteration, a remote sensing image of uniform size and the corresponding extraction truth mask y0 and randomly generated standard Gaussian noise ε are randomly selected, and the relevant timestamp coefficient γ is used to calculate the t , standard Gaussian noise ε and extract the true value mask y0 to generate the noisy target image y t ;

[0016] The remote sensing image of uniform size is used as the conditional image x0 and the related timestamp coefficient γ t and the noisy target image y t , input improved diffusion model, output weighted and noise ε″;

[0017] The loss function in the diffusion model is updated and improved through the loss function formula according to the weighted sum noise ε″ and the standard Gaussian noise ε to complete one iteration.

[0018] By adopting the above technical solution, when training the improved diffusion model, T hyperparameters are determined according to the preset iteration step number T, and then a timestamp t is randomly selected in the interval from 0 to T, and then the timestamp correlation coefficient is generated based on the selected timestamp t and the hyperparameter. In each iteration, a remote sensing image of uniform size and its corresponding extraction truth mask and randomly generated standard Gaussian noise are arbitrarily selected, and the noisy target image is generated using the timestamp correlation coefficient, standard Gaussian noise and extraction truth mask. The uniform size remote sensing image is used as a conditional image, and the timestamp correlation coefficient and the noisy target image are input into the improved diffusion model together, and the model outputs weighted sum noise. Finally, the loss function in the improved diffusion model is updated through the loss function formula according to the weighted sum noise and standard Gaussian noise, and an iterative process is completed. This training method enables the model to acquire the ability to process remote sensing images with different noise levels during the training process, enhances the robustness of the model in processing image noise, and thus improves the reliability of the improved diffusion model in extracting objects in complex environments.

[0019] Optionally, based on the predicted noise ε ′ And standard Gaussian noise ε is updated through the following formula to improve the loss function in the diffusion model,

[0020]

[0021] Among them, w1 and w2 are weighting coefficients, w2=1-w1,ε ′ is the predicted noise, y″ is the predicted target image, y t is the target image with noise.

[0022] By adopting the above technical solution, during the training process, by predicting noise and standard Gaussian noise, and introducing w1 and w2 as weighting coefficients, a loss function formula is established according to the relationship between the noisy target image, the predicted target image and the predicted noise to update the loss function in the improved diffusion model. This loss function calculation method with predicted noise enables the model to not only learn the ability to extract objects during training, but also learn to predict and process image noise that appears in complex environments, thereby improving the model's adaptability to common noise in remote sensing images. This can enhance the robustness of the improved diffusion model on noisy remote sensing data, enabling it to reliably extract objects under complex conditions and obtain more accurate segmentation results.

[0023] Optionally, according to the relevant timestamp coefficient γ t , standard Gaussian noise ε and extract the true value mask y0 to generate the noisy target image y t ,include:

[0024] According to the following formula, the relevant time stamp coefficient γ t , standard Gaussian noise ε and extract the true value mask y0 to generate the noisy target image y t ,

[0025]

[0026] In the formula, y t is the target image with noise, Generate, γ t is the timestamp correlation coefficient, y0 is the target image without noise, and ε is the image noise part.

[0027] By adopting the above technical solution, in the process of generating the noisy target image, the relevant time stamp coefficient γ is used t With standard Gaussian noise ε, according to the formula Generate a noisy target image y t This can improve the model's adaptability to common noise in remote sensing images and enhance the model's anti-noise ability, thereby improving the accuracy of object extraction in complex environments and achieving more refined segmentation results.

[0028] Optionally, output weighted and noise ε″, consisting of:

[0029] For the prediction noise ε ′ and the prediction mask y″ are weighted by the following formula:

[0030]

[0031] In the formula, w2=1-w1, w1 and w2 are weighting coefficients, ε ′is the predicted noise, y″ is the predicted target image, y t is the target image containing noise, and λ is the coefficient that controls the influence of the two outputs on the final result.

[0032] By adopting the above technical means, during the model training process, the predicted noise and predicted mask images are weighted, wherein the weighting coefficients α, β and γ are set to control the importance of different outputs, and the λ coefficient is added to balance the influence of the predicted noise and predicted mask on the final output result. This weighted output method that considers both noise prediction and mask prediction can enable the model to learn the ability to extract the features of objects while taking into account the processing of image noise during training. This can increase the probability of the model correctly extracting objects in the presence of noise interference, enhance the robustness of the model in processing complex remote sensing images, and thus output clear mask images in complex environments, achieving reliable object extraction and segmentation.

[0033] Optionally, the remote sensing image to be processed and the standard normal distribution array are split into a number of image blocks and standard normal distribution arrays at corresponding positions, including:

[0034] The remote sensing image to be processed and the standard normal distribution array are split according to the hardware conditions of the remote sensing image to be processed to obtain several image blocks and the standard normal distribution arrays at corresponding positions, wherein each image block has overlapping areas in the upper, lower, left and right directions, and the overlapping areas have the same size.

[0035] By adopting the above technical solution, the remote sensing image is divided to obtain several image blocks, and the size of each block can be reduced to meet the processing requirements of the model. When the mask images output by each image block are spliced, the overlapping areas between the mask images are matched first to determine the splicing position; then the central area of ​​each mask image is located and intercepted, and only the central part is retained; finally, the splicing is performed according to the determined splicing position to obtain a complete target mask image covering the entire remote sensing image range. In this way, the processing method of determining the splicing position by matching and retaining only the central part for splicing can avoid image distortion caused by edge effects during direct splicing, ensure the fineness and clarity of the target mask image after splicing, and thus realize the segmentation and extraction of large-size remote sensing images.

[0036] Optionally, performing splicing processing on mask images corresponding to the plurality of image blocks to obtain a target mask image includes: matching according to overlapping areas of the mask images corresponding to the plurality of image blocks to determine the splicing positions corresponding to the respective mask images; determining the center point of each mask image, and intercepting the center area of ​​the mask image according to the size of the overlapping area to obtain the intercepted mask image;

[0037] The captured mask images are stitched according to the corresponding stitching positions to obtain the target mask image.

[0038] By adopting the above technical solution, the splicing position is determined according to the overlapping area matching between the mask images corresponding to the multiple image blocks, and the corresponding relationship between the mask images can be accurately obtained to provide a position basis for subsequent splicing. By intercepting the central area of ​​each mask image and then splicing it, the image distortion caused by the mismatch at the edge during direct splicing can be avoided. This mask image splicing method not only realizes the segmentation processing of large-scale remote sensing images and improves the model processing efficiency, but also ensures the fineness and clarity of the mask image after splicing through matching and interception means, and effectively prevents the negative impact of edge effects on image quality. Therefore, it can realize the rapid and accurate object extraction of large-scale remote sensing images, which is conducive to the promotion of the improved diffusion model to the processing of wide-area remote sensing images.

[0039] In a second aspect of the present application, a remote sensing image ground object extraction method system based on an artificial intelligence diffusion model is provided, comprising:

[0040] A training sample data set construction module is used to construct a training sample data set, and the training sample data set includes remote sensing images of uniform size and extraction truth value masks corresponding to the remote sensing images;

[0041] A model building module, used to build an improved diffusion model, the improved diffusion model includes a first UNet network and a second UNet network, wherein the first UNet network is used to predict noise, and the second UNet network is used to predict a mask image;

[0042] The model training module is used to train the improved diffusion model through the training sample data set, using the remote sensing image of uniform size as the conditional image x0 and the related timestamp coefficient γ generated based on the preset number of iteration steps. t and the coefficient γ based on the relevant timestamp t , randomly generated standard Gaussian noise ε and extracted true value mask y0 to generate a noisy target image y t As input, the improved diffusion model is iteratively trained, and when the accuracy of the improved diffusion model reaches a preset standard, the target improved diffusion model is obtained; the data acquisition module is used to obtain the remote sensing image to be processed, and generate a standard normal distribution array based on the remote sensing image to be processed and the preset number of iteration steps;

[0043] A splitting module is used to split the remote sensing image to be processed and the standard normal distribution array into a number of image blocks and the standard normal distribution arrays at corresponding positions, and input the number of image blocks and the standard normal distribution arrays at corresponding positions into the target improved diffusion model to obtain mask images corresponding to the number of image blocks;

[0044] The target mask image stitching module is used to stitch the mask images corresponding to several image blocks to obtain the target mask image.

[0045] In the fourth aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the above method.

[0046] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: 1. The present application constructs a training sample data set containing a remote sensing image of uniform size and a corresponding extracted true value mask, which can provide a complete remote sensing image and expected output information for iterative training of an improved diffusion model. The improved diffusion model includes two subnetworks UNet that independently process noise prediction and mask extraction, and reduces the difficulty of model training and accelerates the convergence speed by decomposing subtasks. During the iterative training process, the Gaussian noise level is controlled according to the timestamp correlation coefficient and a noisy target image is generated with the true value mask, which can effectively enhance the robustness of the model in processing complex noise. After obtaining the trained improved diffusion model, the mask output can be quickly generated for the split remote sensing image block, and then the complete target mask image reflecting the overall distribution of the object can be obtained by splicing. In this way, the improved diffusion model not only improves the quality of object image extraction by decomposing the model construction of the task system and iterative training of noisy images, but also greatly shortens the generation time of the mask image in the remote sensing image object extraction based on the diffusion model, so that the object extraction speed is improved.

[0047] 2. When training the improved diffusion model, the present application determines T hyperparameters according to the preset number of iteration steps T, and then randomly selects a timestamp t in the interval from 0 to T, and then generates a timestamp correlation coefficient based on the selected timestamp t and the hyperparameters. In each iteration, a remote sensing image of uniform size and its corresponding extraction truth mask and randomly generated standard Gaussian noise are arbitrarily selected, and a noisy target image is generated using the timestamp correlation coefficient, standard Gaussian noise and extraction truth mask. The remote sensing image of uniform size is used as a conditional image, and the timestamp correlation coefficient and the noisy target image are input into the improved diffusion model together, and the model outputs weighted noise. Finally, the loss function in the improved diffusion model is updated through the loss function formula based on the weighted sum noise and standard Gaussian noise, and an iterative process is completed. This training method enables the model to acquire the ability to process remote sensing images with different noise levels during the training process, enhances the robustness of the model in processing image noise, and thus improves the reliability of the improved diffusion model in extracting objects in complex environments.

[0048] 3. This application obtains several image blocks by dividing the remote sensing image, which can reduce the size of each block to meet the processing requirements of the model. When splicing the mask images output by each image block, first match the overlapping areas between the mask images to determine their splicing positions; then locate the central area of ​​each mask image and cut it off, retaining only the central part; finally, splice according to the determined splicing position to obtain a complete target mask image covering the entire remote sensing image range. This processing method of determining the splicing position by matching and retaining only the central part for splicing can avoid image distortion caused by edge effects during direct splicing, ensure the fineness and clarity of the target mask image after splicing, and thus achieve the segmentation and extraction of large-size remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flowchart of a method for extracting ground objects from remote sensing images based on an artificial intelligence diffusion model provided in an embodiment of the present application;

[0050] Figure 2 A schematic diagram of image splitting provided in an embodiment of the present application;

[0051] Figure 3 An architecture diagram of a remote sensing image object extraction system based on an artificial intelligence diffusion model provided in an embodiment of the present application;

[0052] Figure 4 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0053] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0054] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0055] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0056] In order to facilitate understanding of the method and system provided by the embodiments of the present application, before introducing the embodiments of the present application, the background of the embodiments of the present application is first introduced.

[0057] In the field of remote sensing, object extraction technology is widely used, such as in urban planning, change detection, mapping and real-time monitoring, and is of great significance to the acquisition of geographic information. However, traditional methods have many difficulties in the extraction process. In recent years, deep learning technology has shown great potential in the task of object extraction and has achieved very good results. At present, semantic segmentation network is the dominant technology for object extraction in remote sensing images. The diffusion model has been well applied in the field of image generation due to its unique advantages in processing noisy images. However, the application of diffusion models in the delicate segmentation task of object extraction in remote sensing images is still in its infancy. This is mainly because the traditional diffusion model requires a large number of iterative steps to introduce Gaussian noise parameters during the extraction process, resulting in a long time to generate the mask image of the remote sensing image. Therefore, studying methods to improve the segmentation speed of the diffusion model in remote sensing image object extraction while maintaining its noise processing ability is an important research direction at present.

[0058] After the background introduction of the above content, those skilled in the art can understand the problems existing in the prior art. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0059] Based on the above background technology, please refer to Figure 1 , Figure 1 A flow chart of a method for extracting ground objects from remote sensing images based on an artificial intelligence diffusion model provided in an embodiment of the present application. The system can be implemented by a computer program or can be run as an independent tool application. Specifically, in an embodiment of the present application, the method can be applied on a server, but can also be applied to electronic devices such as servers. A method for extracting ground objects from remote sensing images based on an artificial intelligence diffusion model includes the following steps:

[0060] S101, constructing a training sample data set, where the training sample data set includes remote sensing images of uniform size and extraction truth value masks corresponding to the remote sensing images;

[0061] Specifically, in order to train the improved diffusion model, it is necessary to first construct a training sample data set. The training sample data set constructed here contains remote sensing images of uniform size and the corresponding extracted true value masks. The purpose of constructing remote sensing image samples of uniform size is to facilitate the batch processing of samples during model training and to improve the speed of model training. For each remote sensing image sample, the true value mask of the focused object is manually drawn as the expected result in the training process. The training sample data set containing remote sensing images of uniform size and the corresponding extracted true value masks constructed in this way can provide a complete information source of uniform specifications including the original image and the expected output for iterative training of the improved diffusion model, making the model training process efficient and stable, thereby obtaining a target improved diffusion model that can extract objects at high speed for different remote sensing images.

[0062] S102, constructing an improved diffusion model, the improved diffusion model comprising a first UNet network and a second UNet network, wherein the first UNet network is used to predict noise, and the second UNet network is used to predict a mask image;

[0063] Specifically, in order to shorten the generation time of the traditional diffusion model in the extraction of ground objects, an improved diffusion model is constructed here. The model consists of two parts. The first part is the UNet network, which is used to predict the noise distribution in the remote sensing image. The second part is the UNet network, which is used to predict the mask image of the ground object. The purpose of constructing this improved diffusion model containing two sub-networks, noise prediction and mask prediction, is to enable the model to have both the ability to predict and process noise and the ability to extract the contour of the ground object. In this way, during the training process, the noise prediction part can enhance the robustness of the model in dealing with complex environments, and the mask prediction part ensures the extraction of accurate ground object contours. The final target improved diffusion model can not only quickly predict noise, but also efficiently output ground object masks. The sub-task decomposition greatly shortens the ground object extraction time, and realizes the improvement of the segmentation speed of the diffusion model in remote sensing image extraction.

[0064] S103, when training the improved diffusion model through the training sample data set, the remote sensing image of uniform size is used as the conditional image x0, and the relevant timestamp coefficient γ generated based on the preset iteration step number t and the coefficient γ based on the relevant timestamp t , randomly generated standard Gaussian noise ε and extracted true value mask y0 to generate a noisy target image y t As input, the improved diffusion model is iteratively trained, and when the accuracy of the improved diffusion model reaches a preset standard, the target improved diffusion model is obtained;

[0065] Specifically, in order to train the constructed improved diffusion model, the training sample data set containing the uniform-sized remote sensing images and the corresponding extracted true value masks constructed in the early stage is used for iterative training. Among them, the uniform-sized remote sensing images are used as the conditional image input of the model; at the same time, the timestamp correlation coefficient is generated based on the preset number of iteration steps, and the coefficient is used to control the addition of standard Gaussian noise, combined with the corresponding extracted true value mask, to synthesize the target image with different levels of noise as the input of the model, and output the predicted noise and mask. Iterative training in this way can enhance the robustness of the model in processing remote sensing images in complex environments, so that it can consider both the contour of the object and the noise distribution during extraction, so as to achieve the effect of accurately extracting the object. When the extraction accuracy of the model on the validation set reaches the preset standard, the target improved diffusion model with both noise resistance and high-speed extraction of the contour of the object is obtained.

[0066] Based on the above embodiment, as an optional embodiment, when training the improved diffusion model through the training sample data set, the remote sensing image of uniform size is used as the conditional image x0, and the relevant timestamp coefficient γ generated based on the preset iteration step is t and the coefficient γ based on the relevant timestamp t , randomly generated standard Gaussian noise ε and extracted true value mask y0 to generate a noisy target image y t As input, the improved diffusion model is iteratively trained, including:

[0067] S201, set the number of hyperparameters α to T according to the preset number of iteration steps T t ;

[0068] Specifically, when training the improved diffusion model, first set a series of hyperparameters α that control the diffusion process. t , α t The value range is between 0 and 1. When training the improved diffusion model, the same number of hyperparameters needs to be set based on the preset number of iterations T. The number of hyperparameters is set to be the same as the number of iterations so that a corresponding hyperparameter value can be obtained in each iteration to calculate the timestamp correlation coefficient required for the iteration. Setting more hyperparameters can provide richer combinations for the timestamp correlation coefficient, thereby enriching the noise level in the target image. Fitting different levels of noise can enhance the model's ability to handle noise. Setting the number of hyperparameters to the number of iterations not only provides sufficient hyperparameter selection, but also avoids the increase in computational complexity caused by too many hyperparameter settings. Therefore, this setting method ensures both the model training effect and computational efficiency.

[0069] S202, randomly select a timestamp t in the interval 0 to T, and calculate the timestamp t according to the hyperparameter αt and timestamp t to generate the related timestamp coefficient γ t ;

[0070] Specifically, during the model training process, each time an iteration is performed, a timestamp t is randomly selected in the interval from 0 to the preset number of iterations T. Then, the timestamp correlation coefficient corresponding to the iteration step is calculated based on the set hyperparameters and the selected timestamp t. The reason for randomly selecting timestamps is to make the noise level of the target image different in each iteration step, so that the model can learn the ability to deal with different levels of noise. During the specific calculation, the hyperparameters related to the selected timestamp t determine the value of the timestamp correlation coefficient, thereby controlling the noise level. By controlling the noise level through random timestamps, the robustness of the model in handling complex noise can also be fully enhanced, so that it can consider the noise distribution in different environments when extracting objects and output more accurate masks.

[0071] In a specific embodiment, a timestamp t is randomly selected in the interval 0 to T to generate a related timestamp coefficient γ t , where the relevant timestamp coefficient γ t , calculated by the following formula: t =∏ i t ＝1 α t .

[0072] S203, in each iteration, arbitrarily select a remote sensing image of uniform size and the extracted truth mask y0 corresponding to the remote sensing image and the randomly generated standard Gaussian noise ε, and calculate the time stamp coefficient γ according to the relevant time stamp coefficient γ. t , standard Gaussian noise ε and extract the true value mask y0 to generate the noisy target image y t ;

[0073] S204, using the remote sensing image of uniform size as the conditional image x0 and the related timestamp coefficient γ t and the noisy target image y t , input improved diffusion model, output weighted and noise ε″;

[0074] Specifically, in each iterative training, the currently selected remote sensing image of uniform size is used as the conditional image, and the corresponding timestamp correlation coefficient and the noisy target image are input into the improved diffusion model. The model will output the predicted weighted and noise results. In this way, the original image and the target image with different levels of noise are input for training at the same time, so that the model can predict the corresponding noise distribution while processing the remote sensing image, so as to achieve accurate positioning of the boundary of the object in the extraction result. As the training progresses, the model can gradually learn how to predict the appropriate noise distribution based on the original image to reduce the interference of noise on the extraction result. The final model can not only process remote sensing images quickly, but also model the noise in complex environments to achieve accurate extraction of objects in remote sensing images.

[0075] Based on the above embodiment, as an optional embodiment, according to the relevant time stamp coefficient γ t , standard Gaussian noise ε and extract the true value mask y0 to generate the noisy target image y t ,include:

[0076] According to the following formula, the relevant time stamp coefficient γ t , standard Gaussian noise ε and extract the true value mask y0 to generate the noisy target image y t ;

[0077] The formula is:

[0078]

[0079] In the formula, y t is the target image with noise, Generate, γ t is the timestamp correlation coefficient, y0 is the target image without noise, and ε is the image noise part.

[0080] S205, completing one iteration by updating the loss function in the improved diffusion model through the loss function formula according to the weighted sum noise ε″ and the standard Gaussian noise ε.

[0081] Specifically, after each iteration of training, the loss of the current iteration is calculated using the loss function formula based on the weighted noise of the model output and the standard Gaussian noise of the original input. Then, through back propagation, the loss value is used to update the loss function in the improved diffusion model. In this way, each iteration updates the model parameters by the difference between the model prediction output and the actual noise, which can make the noise prediction part gradually approach the distribution of the standard noise, so that the model learns to predict the noise in the remote sensing image. After enough iterations, the model can quickly infer the noise of each step in the generation process, and gradually denoise the noise based on the noise to obtain accurate ground object extraction results. Therefore, this training method with noise prediction and parameter update can obtain an improved diffusion model that not only takes into account the influence of noise but also achieves fast and accurate extraction.

[0082] Based on the above embodiment, as an optional embodiment, according to the predicted noise ε ′ And standard Gaussian noise ε is updated through the following formula to improve the loss function in the diffusion model,

[0083]

[0084] Among them, w1 and w2 are weighting coefficients, w2=1-w1,ε ′ is the predicted noise, y″ is the predicted target image, y t is the target image with noise.

[0085] In order to further improve the training efficiency and smooth the loss function proportion in each timestamp, the loss function can be further modified as follows:

[0086]

[0087] S104, obtain the remote sensing image to be processed, and generate a standard normal distribution array based on the remote sensing image to be processed and the preset number of iteration steps; specifically, after obtaining the target improved diffusion model, the model can be applied to extract ground objects from the newly input remote sensing image to be processed. To this end, first obtain the remote sensing image to be processed that needs to be extracted. Then, generate a standard normally distributed random array based on the size of the remote sensing image. Since a normally distributed random array is used in each step of the generation process, unnatural edges are prone to occur when the generated results are spliced. Generating a normally distributed array of the full image size and full steps in advance ensures the consistency of the generation of the overlapping parts of the image blocks, so that the spliced ​​edges are very natural.

[0088] In a specific embodiment, when the size of the remote sensing image to be processed is smaller than a preset threshold, an array of size H×W×T is generated based on the remote sensing image to be processed as a standard normal distribution array, wherein H is the height of the remote sensing image to be processed, and W is the width of the remote sensing image to be processed. If the size of the remote sensing image to be processed is larger than the preset threshold, a uniformly distributed random array of size H×W is first generated based on the remote sensing image to be processed as E2, and then each element therein is used as a seed to generate a normally distributed random array of length T, and each normally distributed random array is merged together to generate a randomly distributed array of size (px+2B)×(py+2B)×T.

[0089] S105, splitting the remote sensing image to be processed and the standard normal distribution array into a number of image blocks and standard normal distribution arrays at corresponding positions, and inputting the number of image blocks and the standard normal distribution arrays at corresponding positions into the target improved diffusion model to obtain mask images corresponding to the number of image blocks;

[0090] Specifically, in order to meet the processing requirements of the improved diffusion model, the large-scale remote sensing image to be processed is first split into several small image blocks. Then these image blocks are input into the pre-trained target improved diffusion model together with the previously generated standard normal distribution array. The model can quickly process these small blocks of images and output the corresponding ground object mask image. The reason for the block processing of the image is that the remote sensing image samples used in the training process of the improved diffusion model are of uniform size, and directly inputting too large images will reduce the processing speed of the model. The block input into the pre-trained model can realize the efficient extraction of large-size images, so that the improved diffusion model can achieve the effect of quickly processing large-scale remote sensing images.

[0091] Based on the above embodiment, as an optional embodiment, the remote sensing image to be processed and the standard normal distribution array are split into a number of image blocks and standard normal distribution arrays at corresponding positions, including:

[0092] The remote sensing image to be processed and the standard normal distribution array are split according to the hardware conditions of the remote sensing image to be processed to obtain several image blocks and the standard normal distribution arrays at corresponding positions, wherein each image block has overlapping areas in the upper, lower, left and right directions, and the overlapping areas have the same size.

[0093] Specifically, in order to meet the model processing requirements, first determine the large-size remote sensing image to be processed into several image blocks suitable for model input based on hardware conditions. Then, based on the size parameters of the remote sensing image and the set number of model iteration steps, generate a standard normal distribution array that matches the size and number of iteration steps of the remote sensing image. The purpose of block processing of remote sensing images is to solve the problem that a single block of remote sensing images is too large to be directly input into the model, so that the model can efficiently process large-scale remote sensing data. The corresponding normal distribution array is generated to meet the parameter requirements in the generation process and ensure natural splicing. Therefore, these two steps of processing allow the improved diffusion model to quickly generate segmentation results for the split remote sensing image blocks and ensure natural splicing.

[0094] It should be noted that hardware conditions mainly refer to the GPU memory size and computing power of the computer / device where model training and inference are performed.

[0095] Splitting large-size remote sensing images into blocks is mainly to solve the problem that the size of a single remote sensing image is too large and exceeds the GPU memory limit and computing load capacity of the model.

[0096] Therefore, when segmenting remote sensing images, the following hardware conditions need to be considered:

[0097] 1. GPU memory size: According to the GPU memory size, determine the maximum size of a single remote sensing image block to adapt to the GPU memory limit.

[0098] 2. GPU computing power: Determine the size of a single remote sensing image block based on the GPU's floating-point computing power to keep the computing time within an acceptable range.

[0099] 3. CPU memory size. The split remote sensing image blocks need to be temporarily stored in the CPU memory, which cannot exceed the size of the CPU memory.

[0100] 4. Storage space. The split image blocks need to have enough disk space to store them.

[0101] After comprehensively considering these hardware conditions, the specific block division scheme of the remote sensing image is determined so that each block is suitable for model loading and processing, thereby achieving efficient processing of large-size remote sensing images.

[0102] Secondly, in order to smoothly stitch the extraction results of each image block in the future, when splitting the remote sensing image to be processed, it is necessary to control the overlap between adjacent image blocks. And here, the overlap between the four sides of each image block and the adjacent blocks is set to the same size. The overlapping area is set in this way to provide a reference for position matching during stitching and determine the relative position relationship between the blocks. If the overlapping area is too small, it is not conducive to matching, and if it is too large, it will increase the amount of redundant calculations. Therefore, setting a moderate overlapping area size that is the same everywhere is convenient for the subsequent accurate stitching of the results of each block, and avoiding too much unnecessary calculation burden. This kind of overlapping area division can enable the improved model to effectively extract the block remote sensing images, and make the results of each block output easy to stitch, thereby achieving the purpose of segmenting a large range of remote sensing images.

[0103] In the specific embodiment, please see Figure 2 , split the remote sensing image to be processed into several image blocks for processing, the image blocks overlap up and down, left and right, the overlap size is 2B, the image size is (px+2B)×(py+2B), px and py are the stitching image size. In this embodiment, a 12G memory TITANXp is used in the application link, px, px takes the value of 128, and B takes the value of 64. The image block size is set to 256x256, the image blocks overlap up and down, left and right, and the geographic coordinates of each block are retained.

[0104] S106, performing splicing processing on mask images corresponding to a plurality of image blocks to obtain a target mask image.

[0105] Specifically, in order to obtain a complete remote sensing image mask, it is necessary to splice the mask images corresponding to several image blocks output by the previous block processing. The splicing process includes: first, matching according to the overlapping areas between the mask images to determine their splicing positions; then intercepting the central area to prevent edge distortion that may occur during direct splicing; finally, splicing according to the determined positional relationship, and finally obtaining a complete target mask image covering the entire remote sensing image range. The mask image is spliced ​​because the original remote sensing image has been processed by blocks, and the block results need to be connected to obtain a mask that reflects the distribution of objects in the entire area. The splicing process can effectively avoid image quality loss at the splicing gaps through matching, interception and reorganization, thereby obtaining a clear and fine target mask image.

[0106] See also Figure 3 , Figure 3 The architecture diagram of a remote sensing image ground object extraction system based on an artificial intelligence diffusion model provided in an embodiment of the present application, the remote sensing image ground object extraction system based on an artificial intelligence diffusion model may include:

[0107] The training sample data set construction module 1 is used to construct a training sample data set, wherein the training sample data set includes remote sensing images of uniform size and extraction truth value masks corresponding to the remote sensing images;

[0108] A model building module 2 is used to build an improved diffusion model, the improved diffusion model includes a first UNet network and a second UNet network, wherein the first UNet network is used to predict noise, and the second UNet network is used to predict a mask image;

[0109] Model training module 3 is used to train the improved diffusion model through the training sample data set, using the remote sensing image of uniform size as the conditional image x0 and the related timestamp coefficient γ generated based on the preset number of iteration steps. t and the coefficient γ based on the relevant timestamp t , randomly generated standard Gaussian noise ε and extracted true value mask y0 to generate a noisy target image y t As input, the improved diffusion model is iteratively trained, and when the accuracy of the improved diffusion model reaches a preset standard, the target improved diffusion model is obtained; the data acquisition module 4 is used to obtain the remote sensing image to be processed, and generate a standard normal distribution array based on the remote sensing image to be processed and the preset number of iteration steps;

[0110] A splitting module 5 is used to split the remote sensing image to be processed and the standard normal distribution array into a number of image blocks and the standard normal distribution arrays at corresponding positions, and input the number of image blocks and the standard normal distribution arrays at corresponding positions into the target improved diffusion model to obtain mask images corresponding to the number of image blocks;

[0111] The target mask image stitching module 6 is used to stitch the mask images corresponding to a plurality of image blocks to obtain a target mask image.

[0112] Please refer to Figure 4 The application also discloses an electronic device. Figure 4 The electronic device 400 may include: at least one processor 401 , at least one network interface 404 , a user interface 403 , a memory 405 , and at least one communication bus 402 .

[0113] The communication bus 402 is used to realize the connection and communication between these components.

[0114] The user interface 403 may include a display screen (Display) and a camera (Camera), and the optional user interface 403 may also include a standard wired interface and a wireless interface.

[0115] The network interface 404 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0116] Among them, the processor 401 may include one or more processing cores. The processor 401 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 405, and calling data stored in the memory 405. Optionally, the processor 401 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field ~ Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 401 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 401, and it can be implemented separately through a chip.

[0117] Among them, the memory 405 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 405 includes a non-transitory computer-readable medium. The memory 405 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 405 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 405 may also be optionally at least one storage system located away from the aforementioned processor 401. Refer to Figure 4 , the memory 405 as a computer storage medium may include an operating system, a network communication module, a user interface module and an application program of a remote sensing image ground object extraction method based on an artificial intelligence diffusion model.

[0118] exist Figure 4In the electronic device 400 shown, the user interface 403 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 401 can be used to call the application program storing the road assessment method in the memory 405. When executed by one or more processors 401, the electronic device 400 executes one or more methods in the above-mentioned embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application. In the above embodiments, the description of each embodiment has its own emphasis. For the parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0119] In the several implementations provided in this application, it should be understood that the disclosed system can be implemented in other ways. For example, the system embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of the system or unit can be electrical or other forms.

[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0121] The present application also provides a computer storage medium that can store multiple instructions, which are suitable for being loaded and executed by a processor as described above. Figure 1 The road assessment method of the embodiment shown in the figure can be found in the specific implementation process. Figure 1 The specific description of the illustrated embodiment will not be repeated here.

[0122] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0123] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0124] The above are only exemplary embodiments of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0125] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for extracting ground objects from remote sensing images based on an artificial intelligence diffusion model, characterized in that: The method comprises: Constructing a training sample data set, wherein the training sample data set includes remote sensing images of uniform size and extraction truth value masks corresponding to the remote sensing images; Constructing an improved diffusion model, the improved diffusion model comprising a first UNet network and a second UNet network, wherein the first UNet network is used to predict noise, and the second UNet network is used to predict a mask image; When the improved diffusion model is trained by the training sample data set, the remote sensing image of uniform size is used as the conditional image x0, and the relevant timestamp coefficient γ generated based on the preset number of iteration steps is used as the conditional image x0. t and based on the associated timestamp coefficient γ t , randomly generated standard Gaussian noise ε and the extracted true value mask y0 generate a noisy target image y t As input, the improved diffusion model is iteratively trained, and when the accuracy of the improved diffusion model reaches a preset standard, a target improved diffusion model is obtained; Acquire a remote sensing image to be processed, and generate a standard normal distribution array based on the remote sensing image to be processed and the preset number of iteration steps; Splitting the remote sensing image to be processed and the standard normal distribution array into a number of image blocks and standard normal distribution arrays at corresponding positions, and inputting the number of image blocks and the standard normal distribution arrays at corresponding positions into the target improved diffusion model to obtain mask images corresponding to the number of image blocks; Performing splicing processing on the mask images corresponding to the plurality of image blocks to obtain a target mask image; According to the predicted noise ε ′ and the standard Gaussian noise ε update the loss function in the improved diffusion model through the following formula: Among them, w1 and w2 are weighting coefficients, λ is the coefficient that controls the influence of the two outputs on the final result, w2=1-w1, ε ′ is the predicted noise, y″ is the predicted mask, and y t is the target image containing noise.

2. The method according to claim 1, characterized in that When the improved diffusion model is trained by the training sample data set, the remote sensing image of uniform size is used as the conditional image x0, and the relevant timestamp coefficient γ generated based on the preset number of iteration steps is used as the conditional image x0. t and based on the associated timestamp coefficient γ t , randomly generated standard Gaussian noise ε and the extracted true value mask y0 generate a noisy target image y t As input, the improved diffusion model is iteratively trained, including: According to the preset number of iterations T, set the number of hyperparameters α to T t ; A timestamp t is randomly selected in the interval 0 to T, and the t and the timestamp t to generate the related timestamp coefficient γ t ; In each iteration, a remote sensing image of uniform size and the extracted truth mask y0 corresponding to the remote sensing image and the randomly generated standard Gaussian noise ε are arbitrarily selected, and the relevant timestamp coefficient γ is used to extract the truth mask y0 and the randomly generated standard Gaussian noise ε. t , the standard Gaussian noise ε and the extracted true value mask y0 generate a noisy target image y t ; The remote sensing image of uniform size is used as the conditional image x0 and the related timestamp coefficient γ t and the noisy target image y t , input the improved diffusion model, output weighted and noise ε″; One iteration is completed by updating the loss function in the improved diffusion model through a loss function formula according to the weighted sum noise ε″ and the standard Gaussian noise ε.

3. The method according to claim 1, characterized in that The time stamp coefficient γ t , the standard Gaussian noise ε and the extracted true value mask y0 generate a noisy target image y t ,include: According to the related time stamp coefficient γ t , the standard Gaussian noise ε and the extracted true value mask y0 generate a noisy target image y t , In the formula, y t is the target image with noise, Generate, γ t is the timestamp correlation coefficient, y0 is the true value mask, and ε is the image noise part.

4. The method according to claim 2, characterized in that: The output weighted sum noise ε″ includes: For the prediction noise ε ′ and the prediction mask y″ are weighted by the following formula: In the formula, w2=1-w1, w1 and w2 are weighting coefficients, ε ′ is the predicted noise, y″ is the predicted mask, and y t is the target image containing noise, and λ is the coefficient that controls the influence of the two outputs on the final result.

5. The method according to claim 1, characterized in that: The step of splitting the remote sensing image to be processed and the standard normal distribution array into a plurality of image blocks and standard normal distribution arrays at corresponding positions includes: The remote sensing image to be processed and the standard normal distribution array are split according to the hardware conditions of the remote sensing image to be processed to obtain a number of image blocks and standard normal distribution arrays at corresponding positions, wherein each image block has overlapping areas in the upper, lower, left and right directions, and the overlapping areas are of the same size.

6. The method according to claim 1, characterized in that The step of performing splicing processing on the mask images corresponding to the plurality of image blocks to obtain a target mask image includes: Matching is performed according to overlapping areas of mask images corresponding to a plurality of the image blocks to determine a splicing position corresponding to each of the mask images; Determine the center point of each of the mask images, and intercept the center area of ​​the mask image according to the size of the overlapping area to obtain a intercepted mask image; The intercepted mask images are spliced ​​according to the corresponding splicing positions to obtain a target mask image.

7. A remote sensing image feature extraction system based on artificial intelligence diffusion model, characterized in that: include: A training sample data set construction module is used to construct a training sample data set, wherein the training sample data set includes remote sensing images of uniform size and extraction truth value masks corresponding to the remote sensing images; A model building module, used to build an improved diffusion model, wherein the improved diffusion model includes a first UNet network and a second UNet network, wherein the first UNet network is used to predict noise, and the second UNet network is used to predict a mask image; A model training module is used to train the improved diffusion model through the training sample data set, using the remote sensing image of uniform size as a conditional image x0, and the related timestamp coefficient γ generated based on a preset number of iteration steps. t and based on the associated timestamp coefficient γ t , randomly generated standard Gaussian noise ε and the extracted true value mask y0 generate a noisy target image y t As input, the improved diffusion model is iteratively trained, and when the accuracy of the improved diffusion model reaches a preset standard, a target improved diffusion model is obtained; A data acquisition module, used for acquiring the remote sensing image to be processed, and generating a standard normal distribution array based on the remote sensing image to be processed and the preset number of iteration steps; A splitting module is used to split the remote sensing image to be processed and the standard normal distribution array into a number of image blocks and standard normal distribution arrays at corresponding positions, and input the number of image blocks and the standard normal distribution arrays at corresponding positions into the target improved diffusion model to obtain mask images corresponding to the number of image blocks; A target mask image stitching module is used to stitch the mask images corresponding to the plurality of image blocks to obtain a target mask image; According to the predicted noise ε ′ and the standard Gaussian noise ε update the loss function in the improved diffusion model through the following formula: Among them, w1 and w2 are weighting coefficients, λ is the coefficient that controls the influence of the two outputs on the final result, w2=1-w1, ε ′ is the predicted noise, y″ is the predicted mask, and y t is the target image containing noise.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: The electronic device comprises a processor, a memory and a transceiver, wherein the memory is used to store instructions, the transceiver is used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 6.

Images