A remote sensing satellite on-board image AI prediction method and system

Abstract

The present invention provides an AI prediction method and system for images on a remote sensing satellite, which relates to the field of image processing technology. The method includes: acquiring sensor data; intelligently predicting the environmental state of the remote sensing satellite based on the gated cyclic unit according to the sensor data; intelligently predicting the remote sensing predicted image when the remote sensing satellite performs shooting based on the generative adversarial network according to the predicted environmental state; determining the image quality score of the remote sensing predicted image based on the deep learning network according to the remote sensing predicted image; correcting the image quality score obtained based on the deep learning network according to the grayscale average gradient and signal-to-noise ratio of the remote sensing predicted image; judging whether the image quality score of the remote sensing predicted image is greater than a preset score; if so, executing a shooting instruction; otherwise, refusing to execute the shooting instruction. The present invention can avoid the invalidity of the remote sensing images shot due to environmental factors such as weather, and significantly improve the resource utilization effect of the remote sensing satellite.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an AI prediction method and system for images on a remote sensing satellite. Background Art

[0002] With the development of science and technology, remote sensing satellites are playing an increasingly important role in astronomical observation, climate monitoring, geographical research and other fields. Remote sensing images can be used to observe and study celestial bodies in the universe, such as galaxies, stars, planets and asteroids, helping scientists understand the structure and evolution of the universe. Remote sensing images can be used to monitor atmospheric conditions, cloud distribution, rainfall, etc., which helps to improve the accuracy of weather forecasts. Through long-term series of remote sensing images, global climate change trends, such as rising temperatures and frequent extreme weather events, can be analyzed to provide data support for climate change research. Remote sensing images can accurately map the topography of the earth's surface, including mountains, rivers, lakes, etc., providing important data for geological research.

[0003] However, remote sensing satellites have limited energy and shooting opportunities are precious. If the remote sensing images taken are invalid due to environmental factors such as weather, it will cause a huge waste of satellite resources. At the same time, invalid images also occupy storage and transmission resources, increasing the burden of data processing. Summary of the invention

[0004] In order to solve the technical problems in the prior art that the energy of remote sensing satellites is limited and shooting opportunities are precious, if the remote sensing images taken are invalid due to environmental factors such as weather, it will cause a huge waste of satellite resources. At the same time, invalid images also occupy storage and transmission resources, increasing the burden of data processing, the present invention provides an AI prediction method and system for on-board images of remote sensing satellites.

[0005] The technical solution provided by the embodiment of the present invention is as follows:

[0006] First aspect:

[0007] An embodiment of the present invention provides an AI prediction method for images on a remote sensing satellite, comprising:

[0008] S1: Get sensor data;

[0009] S2: intelligently predicting the environmental state of the remote sensing satellite based on the sensor data and the gated recurrent unit;

[0010] S3: Based on the predicted environmental state and the generative adversarial network, the remote sensing predicted image when the remote sensing satellite is shooting is intelligently predicted;

[0011] S4: Determine an image quality score of the remote sensing predicted image based on a deep learning network according to the remote sensing predicted image;

[0012] S5: Correcting the image quality score obtained based on the deep learning network according to the grayscale average gradient and the signal-to-noise ratio of the remote sensing prediction image;

[0013] S6: Determine whether the image quality score of the remote sensing predicted image is greater than a preset score; if so, execute the shooting instruction; otherwise, refuse to execute the shooting instruction.

[0014] Second aspect:

[0015] An embodiment of the present invention provides an AI prediction system for images on a remote sensing satellite, including:

[0016] processor;

[0017] A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the remote sensing satellite on-board image AI prediction method as described in the first aspect is implemented.

[0018] The third aspect:

[0019] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the remote sensing satellite on-board image AI prediction method as described in the first aspect is implemented.

[0020] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0021] In the present invention, the environmental state of the remote sensing satellite is predicted, and then AI technology is used to predict the remote sensing predicted image when the remote sensing satellite is performing shooting. By judging the image quality of the remote sensing predicted image, it is decided whether to perform shooting. This can avoid the invalidity of the remote sensing images taken due to environmental factors such as weather, improve the quality of remote sensing images, significantly improve the resource utilization effect of the remote sensing satellite, avoid invalid images occupying storage and transmission resources, reduce the data processing burden, and reduce satellite communication costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A schematic diagram of a flow chart of an AI prediction method for images on a remote sensing satellite provided by an embodiment of the present invention;

[0024] Figure 2A schematic diagram of the structure of an AI prediction method for images on a remote sensing satellite provided by an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of the structure of an AI prediction system for onboard images of a remote sensing satellite provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0027] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0028] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0029] In the embodiments of the present invention, sometimes the subscripts such as W 1 It may be mistakenly written as a non-subscript form such as W1. When the difference is not emphasized, the meanings they express are the same.

[0030] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0031] Reference Manual Attached Figure 1 , showing a flow chart of an AI prediction method for on-board images of a remote sensing satellite provided in an embodiment of the present invention.

[0032] Reference Manual Attached Figure 2 , showing a structural schematic diagram of an AI prediction method for on-board images of a remote sensing satellite provided by an embodiment of the present invention.

[0033] The embodiment of the present invention provides an AI (Artificial Intelligence) prediction method for remote sensing satellite images, which can be implemented by an AI prediction device for remote sensing satellite images, and the AI ​​prediction device for remote sensing satellite images can be a terminal or a server. The processing flow of the AI ​​prediction method for remote sensing satellite images can include the following steps:

[0034] S1: Get sensor data.

[0035] Among them, sensor data includes atmospheric temperature, humidity, wind speed, wind direction, rainfall, cloud cover, light intensity, air pressure, aerosol concentration, etc.

[0036] S2: Based on sensor data and gated recurrent units, intelligently predict the environmental status of remote sensing satellites.

[0037] Among them, the Gated Recurrent Unit (GRU) is an improved recurrent neural network (RNN) structure that is used to solve the gradient vanishing and gradient exploding problems of traditional RNN when processing long sequence data. GRU introduces a gating mechanism to control the flow of information, thereby more effectively capturing long-term dependencies in time series data.

[0038] Although the traditional gated recurrent unit alleviates the gradient vanishing and gradient exploding problems through the gating mechanism, it may still be difficult to capture long-range dependencies when processing very long sequences. In long time sequences, important information may be submerged in a large amount of historical data, making it difficult for the model to focus on information at critical moments. To this end, the present invention provides a new gated recurrent unit in which an attention mechanism is introduced.

[0039] In a possible implementation, the gated recurrent unit includes: an input layer, an update gate, a reset gate, a candidate layer, and a prediction layer, and S2 specifically includes:

[0040] S201: Input sensor sequence data.

[0041] S202: According to the sensor sequence data, based on the gated recurrent unit, extract the hidden state in the sensor sequence data:

[0042] z t =σ(W uz x t +W hz h t-1 +b z )

[0043] r t =σ(W ur x t +Whr h t-1 +b r )

[0044] c t =σ(W uc x t +W hc (r t ⊙h t-1 )+b c )

[0045] h t =(1-z t )⊙h t-1 +z t ⊙c t

[0046] Among them, z t represents the output vector of the update gate at time t, σ() represents the activation function, and W uz represents the weight matrix between the input layer and the update gate, x t represents the sensor sequence data at time t, W hz represents the self-connection weight matrix of the update gate between time t and time t-1, h t-1 represents the hidden state vector at time t-1, b z represents the bias term of the update gate, r t represents the output vector of the reset gate at time t, W ur represents the weight matrix between the input layer and the reset gate, W hr represents the self-connection weight matrix between the reset gate at time t and time t-1, b r represents the bias term of the reset gate, c t represents the output vector of the candidate layer at time t, W uc Represents the weight matrix between the input layer and the candidate layer, W hc represents the self-connection weight matrix between the candidate layer at time t and time t-1, b c represents the bias term of the candidate layer, ⊙ represents the element-wise product operation, and h t Represents the hidden state vector at time t.

[0047] S203: Add attention weights to the hidden states at each moment:

[0048] e t =σ(W h h t +b h )

[0049]

[0050] h t '=αt h t

[0051] Among them, e t represents the attention score at time t, σ() represents the activation function, and W h represents the attention weight matrix, h t represents the hidden state vector at time t, b h represents the bias term of the attention layer, α t represents the attention weight at time t, exp() represents an exponential function with a natural constant as the base, T represents the total length of the data, and h t ' represents the hidden state vector after adding the attention weight.

[0052] It should be noted that adding attention weights to the hidden states at each moment can dynamically adjust the importance of information at different time steps, focus on the moments that contribute most to the prediction of the current environmental state, and enhance the accuracy and robustness of the prediction.

[0053] S204: According to the hidden state vector after adding the attention weight, intelligently predict the environmental state of the remote sensing satellite at the next moment:

[0054]

[0055] in, represents the predicted value of sensor data at time t+1, σ() represents the activation function, and W x represents the prediction weight matrix, h′ t represents the hidden state vector after adding the attention weight, b x Represents the bias term of the prediction layer.

[0056] In the present invention, GRU and attention mechanism enable the model to adaptively learn complex patterns and changes in sensor data, enhance the adaptability of the model in different environments and conditions, and reduce the impact of environmental changes on prediction results. It can significantly improve the accuracy and real-time performance of environmental state prediction, optimize the shooting plan of remote sensing satellites, improve resource utilization efficiency, and ultimately ensure the acquisition of high-quality remote sensing images.

[0057] S3: Based on the predicted environmental state and the generative adversarial network, the remote sensing predicted image when the remote sensing satellite performs shooting is intelligently predicted.

[0058] In a possible implementation, the generative adversarial network includes: a generator and a discriminator. S3 specifically includes:

[0059] S301: Generate multiple remote sensing prediction images through a generator according to multiple environmental conditions.

[0060] S302: Fix the parameters of the generator, take the minimum discriminator training loss function as the goal, and train the discriminator separately through the gradient descent method.

[0061] Among them, the gradient descent method is an iterative algorithm for optimizing functions, which is widely used in the training process of machine learning and deep learning models. It minimizes the loss function to make the model's prediction results as close to the true value as possible.

[0062] Optionally, the discriminator training loss function is specifically:

[0063]

[0064] Among them, L(θ D ) represents the discriminator training loss function, θ D represents the discriminator parameters, z i represents the i-th real sample that obeys the prior distribution, represents the i-th pseudo sample generated by the generator, log represents the logarithmic function with base 10, and m represents the total number of samples.

[0065] S303: Fix the parameters of the discriminator, take the generator training loss function as the minimum as the goal, and train the generator separately through the gradient descent method.

[0066] Optionally, the generator training loss function is specifically:

[0067]

[0068] Among them, L(θ G ) represents the generator training loss function, θ G Represents the generator parameters, y i represents the true label value of the i-th sample, represents the discrimination result of the discriminator for the i-th sample, log represents the logarithmic function with base 10, m represents the total number of samples, α represents the first regularization coefficient, and |||| represents the modulo operation.

[0069] It should be noted that the generative adversarial network can generate very realistic remote sensing images through adversarial training of the generator and the discriminator. The generator is continuously optimized to deceive the discriminator, and the discriminator continuously improves its ability to distinguish between real images and forged images, ultimately significantly improving the quality of the generated images.

[0070] S304: Generate a remote sensing prediction image based on the predicted environmental state through the trained generator.

[0071] Specifically, a deep convolutional neural network (DCNN) can be used to construct the generator and the discriminator.

[0072] In the present invention, through the trained generator, high-quality remote sensing prediction images can be quickly generated according to the predicted environmental conditions, meeting the needs in the real-time environment and improving the response speed.

[0073] S4: Determine the image quality score of the remote sensing predicted image based on the deep learning network according to the remote sensing predicted image.

[0074] In a possible implementation, the deep learning network includes: a convolutional layer, a first fully connected layer, a pooling layer, and a second fully connected layer, and S4 specifically includes:

[0075] S401: Extracting image features of multiple sample remote sensing images through a convolutional layer.

[0076] S402: Obtain residual features between the input sample remote sensing image and the feature image extracted by the convolution layer through the first fully connected layer.

[0077] S403: With the goal of minimizing the residual features obtained by the fully connected layer, the convolution layer is trained by the gradient descent method.

[0078] Optionally, S403 specifically includes:

[0079] Construct the convolutional layer training loss function:

[0080]

[0081] Among them, L l represents the convolutional layer training loss function of the lth convolutional layer, W l represents the weight matrix of the lth convolutional layer, b l represents the bias term of the lth convolutional layer, represents the i-th sample remote sensing image input in the l-th convolutional layer, N represents the total number of sample remote sensing images, represents the convolution function of the lth convolutional layer, Φ represents the kernel function, represents the linear fitting function, |||| represents the modulo operation, and γ represents the second regularization coefficient.

[0082] It should be noted that by minimizing the residual features obtained by the fully connected layer, it can ensure that the features extracted by the convolutional layer are more accurate, reduce information loss and errors, and enable the extracted features to better represent the key information of the input image.

[0083] With the goal of minimizing the convolutional layer training loss function, the convolutional layer is trained using the gradient descent method.

[0084] Specifically, the network parameters of the convolutional layer can be optimized according to the following formula:

[0085]

[0086] Among them, θ k+1 represents the network parameters of the convolutional layer at the k+1th iteration, θ k represents the network parameters of the convolutional layer at the kth iteration, η k represents the adaptive learning rate at the kth iteration, Represents the Huber loss function L for the network parameter θ k is the gradient of , and β represents the momentum coefficient.

[0087] In the present invention, the convergence speed of gradient descent can be accelerated by introducing the momentum term, especially in loss functions with high curvature, long valleys or flat areas. This can effectively reduce oscillations and make parameter updates smoother.

[0088] Furthermore, the adaptive learning rate is calculated as:

[0089]

[0090] Among them, η k represents the adaptive learning rate at the kth iteration, η 0 represents the initial learning rate, The loss function L is related to the network parameters θ u The gradient of , u=1,2,…,k, μ represents a hyperparameter that prevents the denominator from being zero.

[0091] In the present invention, in the initial stage of training, the accumulated sum of squared gradients is small, so the learning rate is relatively large. This allows the model to quickly update parameters and quickly drop to the lower region of the loss function. As training proceeds, the cumulative sum of squared gradients increases and the learning rate gradually decreases. This helps to make more detailed parameter adjustments in the later stages of training to avoid skipping the optimal solution or causing unnecessary oscillations. The adaptive learning rate can be dynamically adjusted during the training process to prevent oscillations caused by excessive learning rates and slow training speeds caused by too small learning rates, thereby preventing overfitting and underfitting to a certain extent.

[0092] It should be noted that through this optimization strategy, it can be ensured that the features extracted by the convolutional layer are more accurate and effective, and the performance of the entire deep learning network in the remote sensing image quality assessment task can be improved, ultimately improving the quality and application value of remote sensing images.

[0093] S404: extracting image features of the remote sensing prediction image through the trained multiple convolutional layers.

[0094] S405: Through the pooling layer, the image features of the remote sensing prediction image extracted by multiple convolutional layers are pooled.

[0095] S406: Through the second fully connected layer, the image features of the remote sensing prediction image after the pooling process are fused and summarized.

[0096] Optionally, S406 specifically includes:

[0097] According to the following formula, the image features of the remote sensing prediction image after pooling are fused and summarized:

[0098]

[0099] Among them, U represents the aggregated image features, u l represents the lth image feature after pooling, ω l represents the weight of the lth image feature, L represents the total number of image features, and also represents the total number of convolutional layers.

[0100] S407, performing a quality score on the remote sensing prediction image based on the fused and summarized image features.

[0101] Optionally, S407 specifically includes:

[0102] The quality of the remote sensing prediction image is scored according to the following formula:

[0103] S = Sigmoid(W u U+b u )

[0104] Among them, S represents the image quality score of the remote sensing prediction image, Sigmoid() represents the Sigmoid activation function, U represents the summarized image features, and W u represents the quality score weight matrix, b u Represents the quality score bias term.

[0105] In the present invention, the image quality of remote sensing prediction images is determined based on a deep learning network, which can significantly improve the accuracy, efficiency and reliability of the evaluation, reduce human intervention, adapt to a wide range of application scenarios, provide rich evaluation information, and have the ability of continuous optimization and expansion.

[0106] S5: According to the grayscale average gradient and signal-to-noise ratio of the remote sensing predicted image, the image quality score obtained based on the deep learning network is corrected.

[0107] It should be noted that although deep learning models perform well in image quality assessment, they may ignore some quality features that traditional methods can accurately capture, such as grayscale average gradient and signal-to-noise ratio. By introducing these traditional indicators, the shortcomings and limitations of deep learning models can be compensated.

[0108] In a possible implementation, S5 is specifically:

[0109] S * =λ 1 S+λ 2 G+λ 3 SNR

[0110] λ 1 +λ 2 +λ 3 =1,λ 1 >λ 2 ,λ 1 >λ 3

[0111] Among them, S* represents the corrected image quality score of the remote sensing prediction image, S represents the image quality score of the remote sensing prediction image, and λ 1 represents the correction coefficient of the image quality score of the remote sensing prediction image, G represents the grayscale average gradient, λ 2 represents the correction coefficient of the grayscale average gradient, SNR represents the signal-to-noise ratio, λ 3 Represents the correction factor for the signal-to-noise ratio.

[0112] Among them, those skilled in the art can set the correction coefficient λ of the image quality score of the remote sensing prediction image according to actual conditions. 1 , correction coefficient λ of grayscale average gradient 2 and the correction factor λ of the signal-to-noise ratio 3 The present invention does not limit the size.

[0113] The grayscale average gradient is specifically:

[0114]

[0115] Among them, G represents the grayscale average gradient, (m,n) represents the pixel coordinates in the remote sensing prediction image, M represents the image length, and N represents the image height. represents the horizontal gradient, represents the vertical gradient, and T represents the remote sensing prediction image.

[0116] The signal-to-noise ratio is:

[0117]

[0118] Among them, SNR represents signal-to-noise ratio, lg represents the logarithmic function with base 10, and N represents the noise image corresponding to the remote sensing prediction image.

[0119] In the present invention, grayscale average gradient and signal-to-noise ratio are introduced as supplementary indicators, which can comprehensively evaluate image quality from multiple angles. The score of the deep learning network may be mainly based on specific features, while grayscale average gradient and signal-to-noise ratio provide additional information to make up for the shortcomings of a single evaluation method. Through this multi-dimensional, multi-indicator comprehensive evaluation method, the accuracy, comprehensiveness and credibility of remote sensing image quality assessment can be significantly improved, resource utilization can be optimized, the quality of shooting decisions can be improved, and ultimately the practical application value of remote sensing images can be improved.

[0120] S6: Determine whether the image quality score of the remote sensing prediction image is greater than a preset score; if so, execute the shooting instruction; otherwise, refuse to execute the shooting instruction.

[0121] Among them, those skilled in the art can set the size of the preset score according to actual conditions, and the present invention does not limit it.

[0122] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0123] In the present invention, the environmental state of the remote sensing satellite is predicted, and then AI technology is used to predict the remote sensing predicted image when the remote sensing satellite is performing shooting. By judging the image quality of the remote sensing predicted image, it is decided whether to perform shooting. This can avoid the invalidity of the remote sensing images taken due to environmental factors such as weather, improve the quality of remote sensing images, significantly improve the resource utilization effect of the remote sensing satellite, avoid invalid images occupying storage and transmission resources, reduce the data processing burden, and reduce satellite communication costs.

[0124] Reference Manual Attached Figure 3 , showing a structural schematic diagram of a remote sensing satellite on-board image AI prediction system provided by the present invention.

[0125] The present invention also provides a remote sensing satellite on-board image AI prediction system 20, which is applied to the above-mentioned remote sensing satellite on-board image AI prediction method, comprising:

[0126] Processor 201;

[0127] The memory 202 stores computer-readable instructions, and when the computer-readable instructions are executed by the processor 201, the remote sensing satellite on-board image AI prediction method as described in the method embodiment is implemented.

[0128] The remote sensing satellite on-board image AI prediction system 20 provided by the present invention can execute the above-mentioned remote sensing satellite on-board image AI prediction method and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0129] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0130] In the present invention, the environmental state of the remote sensing satellite is predicted, and then AI technology is used to predict the remote sensing predicted image when the remote sensing satellite is performing shooting. By judging the image quality of the remote sensing predicted image, it is decided whether to perform shooting. This can avoid the invalidity of the remote sensing images taken due to environmental factors such as weather, improve the quality of remote sensing images, significantly improve the resource utilization effect of the remote sensing satellite, avoid invalid images occupying storage and transmission resources, reduce the data processing burden, and reduce satellite communication costs.

[0131] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0132] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0133] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0134] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0135] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0136] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0137] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0138] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0139] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0140] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0142] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0143] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the remote sensing satellite on-board image AI prediction method as described in the method embodiment.

[0144] A computer-readable storage medium provided by the present invention can implement the steps and effects of the remote sensing satellite on-board image AI prediction method of the above-mentioned method embodiment. To avoid repetition, the present invention will not go into details.

[0145] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0146] In the present invention, the environmental state of the remote sensing satellite is predicted, and then AI technology is used to predict the remote sensing predicted image when the remote sensing satellite is performing shooting. By judging the image quality of the remote sensing predicted image, it is decided whether to perform shooting. This can avoid the invalidity of the remote sensing images taken due to environmental factors such as weather, improve the quality of remote sensing images, significantly improve the resource utilization effect of the remote sensing satellite, avoid invalid images occupying storage and transmission resources, reduce the data processing burden, and reduce satellite communication costs.

[0147] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0148] There are a few points to note:

[0149] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention, and other structures may refer to the general design.

[0150] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0151] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0152] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A remote sensing satellite on-board image AI prediction method, characterized in that: include: S1: Get sensor data; S2: intelligently predicting the environmental state of the remote sensing satellite based on the sensor data and the gated recurrent unit; S3: Based on the predicted environmental state and the generative adversarial network, the remote sensing predicted image when the remote sensing satellite is shooting is intelligently predicted; S4: Determine an image quality score of the remote sensing predicted image based on a deep learning network according to the remote sensing predicted image; S5: Correcting the image quality score obtained based on the deep learning network according to the grayscale average gradient and the signal-to-noise ratio of the remote sensing prediction image; S6: Determine whether the image quality score of the remote sensing predicted image is greater than a preset score; if so, execute the shooting instruction; otherwise, refuse to execute the shooting instruction.

2. The remote sensing satellite on-board image AI prediction method according to claim 1, characterized in that: The gated recurrent unit includes: an input layer, an update gate, a reset gate, a candidate layer and a prediction layer, and S2 specifically includes: S201: Input sensor sequence data; S202: Extracting hidden states in the sensor sequence data based on a gated recurrent unit according to the sensor sequence data: z t =σ(W uz x t +W hz h t-1 +b z ) r t =σ(W ur x t +W hr h t-1 +b r ) c t =σ(W uc x t +W hc (r t ⊙h t-1 )+b c ) h t =(1-z t )⊙h t-1 +z t ⊙c t Among them, z t represents the output vector of the update gate at time t, σ() represents the activation function, and W uz represents the weight matrix between the input layer and the update gate, x t represents the sensor sequence data at time t, W hz represents the self-connection weight matrix of the update gate between time t and time t-1, h t-1 represents the hidden state vector at time t-1, b z represents the bias term of the update gate, r t represents the output vector of the reset gate at time t, W ur represents the weight matrix between the input layer and the reset gate, W hr represents the self-connection weight matrix between the reset gate at time t and time t-1, b r represents the bias term of the reset gate, c t represents the output vector of the candidate layer at time t, W uc Represents the weight matrix between the input layer and the candidate layer, W hc represents the self-connection weight matrix between the candidate layer at time t and time t-1, b c represents the bias term of the candidate layer, ⊙ represents the element-wise product operation, and h t Represents the hidden state vector at time t; S203: Add attention weights to the hidden states at each moment: e t =σ(W h h t +b h ) h′ t =a t h t Among them, e t represents the attention score at time t, σ() represents the activation function, and W h represents the attention weight matrix, h t represents the hidden state vector at time t, b h represents the bias term of the attention layer, α t represents the attention weight at time t, exp() represents an exponential function with a natural constant as the base, T represents the total length of the data, and h t ' represents the hidden state vector after adding the attention weight; S204: According to the hidden state vector after adding the attention weight, intelligently predict the environmental state of the remote sensing satellite at the next moment: in, represents the predicted value of sensor data at time t+1, σ() represents the activation function, and W x represents the prediction weight matrix, h′ t represents the hidden state vector after adding the attention weight, b x Represents the bias term of the prediction layer.

3. The remote sensing satellite on-board image AI prediction method according to claim 1, characterized in that: The generative adversarial network includes: a generator and a discriminator; S3 specifically includes: S301: Generate multiple remote sensing prediction images through a generator according to multiple environmental states; S302: Fix the parameters of the generator, take the minimum discriminator training loss function as the goal, and train the discriminator separately through the gradient descent method; S303: Fix the parameters of the discriminator, take the minimum training loss function of the generator as the goal, and train the generator separately through the gradient descent method; S304: Generate a remote sensing prediction image based on the predicted environmental state through the trained generator.

4. The remote sensing satellite on-board image AI prediction method according to claim 3 is characterized in that: The discriminator training loss function is specifically: Among them, L(θ D ) represents the discriminator training loss function, θ D represents the discriminator parameters, z i represents the i-th real sample that obeys the prior distribution, represents the i-th pseudo sample generated by the generator, log represents the logarithmic function with base 10, and m represents the total number of samples; The generator training loss function is specifically: Among them, L(θ G ) represents the generator training loss function, θ G Represents the generator parameters, y i represents the true label value of the i-th sample, represents the discrimination result of the discriminator for the i-th sample, log represents the logarithmic function with base 10, m represents the total number of samples, α represents the first regularization coefficient, and |||| represents the modulo operation.

5. The remote sensing satellite on-board image AI prediction method according to claim 1, characterized in that: The deep learning network includes: a convolutional layer, a first fully connected layer, a pooling layer and a second fully connected layer, and S4 specifically includes: S401: extracting image features of multiple sample remote sensing images through a convolutional layer; S402: Obtaining residual features between the input sample remote sensing image and the feature image extracted by the convolution layer through the first fully connected layer; S403: training the convolution layer by a gradient descent method with the goal of minimizing the residual features obtained by the fully connected layer; S404: extracting image features of the remote sensing prediction image respectively through the trained multiple convolutional layers; S405: performing pooling processing on the image features of the remote sensing prediction image extracted by multiple convolutional layers through a pooling layer; S406: fusing and summarizing the image features of the remote sensing prediction image after the pooling process through a second fully connected layer; S407, performing a quality score on the remote sensing predicted image according to the fused and summarized image features to determine the image quality of the remote sensing predicted image.

6. The remote sensing satellite on-board image AI prediction method according to claim 5, characterized in that: The S403 specifically includes: Construct the convolutional layer training loss function: Among them, L l represents the convolutional layer training loss function of the lth convolutional layer, W l represents the weight matrix of the lth convolutional layer, b l represents the bias term of the lth convolutional layer, represents the i-th sample remote sensing image input in the l-th convolutional layer, N represents the total number of sample remote sensing images, and f c l represents the convolution function of the lth convolutional layer, Φ represents the kernel function, represents the linear fitting function, |||| represents the modulo operation, and γ represents the second regular coefficient; The convolutional layer is trained by a gradient descent method with the goal of minimizing the convolutional layer training loss function.

7. The remote sensing satellite on-board image AI prediction method according to claim 5, characterized in that: The S406 specifically includes: According to the following formula, the image features of the remote sensing prediction image after pooling are fused and summarized: Among them, U represents the aggregated image features, u l represents the lth image feature after pooling, ω l represents the weight of the lth image feature, L represents the total number of image features, and also represents the total number of convolutional layers.

8. The remote sensing satellite on-board image AI prediction method according to claim 5, characterized in that: The S407 specifically includes: The quality of the remote sensing prediction image is scored according to the following formula: S=Sigmoid(W u U+b u ) Among them, S represents the image quality score of the remote sensing prediction image, Sigmoid() represents the Sigmoid activation function, U represents the summarized image features, and W u represents the quality score weight matrix, b u Represents the quality score bias term.

9. The remote sensing satellite on-board image AI prediction method according to claim 8, characterized in that: The S5 is specifically: S * =λ1S+λ2G+λ3SNR λ1+λ2+λ3=1,λ1>λ2,λ1>λ3 Among them, S* represents the corrected image quality score of the remote sensing prediction image, S represents the image quality score of the remote sensing prediction image, λ1 represents the correction coefficient of the image quality score of the remote sensing prediction image, G represents the grayscale average gradient, λ2 represents the correction coefficient of the grayscale average gradient, SNR represents the signal-to-noise ratio, and λ3 represents the correction coefficient of the signal-to-noise ratio.

10. A remote sensing satellite on-board image AI prediction system, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the remote sensing satellite on-board image AI prediction method as described in any one of claims 1 to 9 is implemented.

Images