Federal learning-based remote sensing image processing method and system, and related equipment

Abstract

The invention is suitable for the technical field of image processing, and provides a remote sensing image processing method and system based on federated learning, and related equipment, and the method comprises the steps: obtaining remote sensing image data by each participant, and carrying out the preprocessing of the data; each participant performs local model training based on the preprocessed data, extracts initial parameters based on a trained model, encrypts the initial parameters, and sends the encrypted initial parameters to an aggregation end; the aggregation end decrypts the received ciphertext to obtain an initial parameter, determines an optimal correction scheme by using a multi-objective optimization algorithm, and corrects the remote sensing image data of each participant according to the optimal correction scheme; extracting a target feature vector set according to the corrected remote sensing image data, and aggregating the target feature vector set through a Bayesian model to generate a target parameter; and the target parameters are distributed to the participants, so that the participants update the local model and execute multiple iterative operations, and the image processing efficiency and accuracy are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a remote sensing image processing method, system and related equipment based on federated learning. Background Art

[0002] Remote sensing is a non-contact detection technology that uses sensors to obtain information about targets on the earth's surface from a long distance, such as from aircraft, satellites and other platforms. With the development of aerospace technology and sensor technology, remote sensing has become an important means of obtaining geospatial information. For example, artificial satellites carry various sensors in orbit in space and can observe the earth regularly or irregularly.

[0003] At present, in view of the difficulties in many aspects of remote sensing image processing methods, some studies use federated learning technology to protect the privacy of remote sensing images. Patent No. CN113239023A discloses a federated learning model training method for remote sensing data, which determines the first feature label library and the first remote sensing data set of the first node, trains the first feature label library based on the remote sensing data set to obtain the first remote sensing federated learning model, and sends it to the second node. After receiving it, the second node trains the second remote sensing data set to obtain the second encrypted remote sensing federated learning model, corrects the first and second remote sensing federated learning model parameters and calculates the encrypted loss value, sends the encrypted loss value to the third node and determines whether it converges, so as to obtain the final parameters or enter the training model calibration. It can be seen that the patent does not disclose the process of how to perform correction after entering the training model, and it is impossible to evaluate the errors caused by various correction factors, so that the remote sensing image data processed during the training process has inherent defects, and it is easy to have the problem of poor image quality in the final output. Summary of the invention

[0004] The embodiments of the present application provide a remote sensing image processing method, system and related equipment based on federated learning, which are used to solve the problem of poor output image quality.

[0005] A first aspect of an embodiment of the present application provides a remote sensing image processing method based on federated learning, comprising:

[0006] Each participant obtains remote sensing image data through several channels and pre-processes the obtained remote sensing image data;

[0007] Each participant performs local model training based on the preprocessed data, extracts initial parameters based on the trained model, encrypts the initial parameters, and sends the encrypted initial parameters to the aggregation end;

[0008] The aggregator decrypts the received ciphertext to obtain the initial parameters, determines an optimal correction scheme based on the initial parameters using a multi-objective optimization algorithm, and corrects the remote sensing image data of each participant according to the optimal correction scheme;

[0009] The aggregation end extracts a target feature vector set according to the corrected remote sensing image data, aggregates the target feature vector set through a Bayesian model, and generates target parameters;

[0010] The aggregation end distributes the target parameters to the various participants, so that the various participants use the target parameters to update the local model and perform multiple iteration operations.

[0011] Furthermore, the aggregator decrypts the received ciphertext to obtain the initial parameters, determines an optimal correction scheme based on the initial parameters using a multi-objective optimization algorithm, and corrects the remote sensing image data of each participant according to the optimal correction scheme, including:

[0012] The aggregation end stores the initial parameters in a preset data structure respectively based on the correction task type of the initial parameters;

[0013] Determine the corresponding objective function based on each correction task type;

[0014] The multi-objective optimization algorithm is used to iteratively optimize the target values ​​corresponding to each correction task type to obtain the target frontier solution set;

[0015] An optimal correction scheme is determined according to the target frontier solution set.

[0016] Furthermore, the determining of the corresponding objective function based on each correction task type includes:

[0017] The various correction task types include radiation correction, geometric correction, atmospheric correction and terrain correction;

[0018] Objective function J of radiation correction rad expression:

[0019]

[0020] Where: n is the number of samples, is the corrected radiation value, y i is the actual radiation value;

[0021] Objective function J of geometric correction geo expression:

[0022]

[0023] Where: m is the number of ground control points, (X j , Y j ) is the actual geographic coordinate, (x j ,y j ) are the image coordinates after geometric correction;

[0024] Objective function of atmospheric correction J atm expression:

[0025]

[0026] Where: k is the number of sample points for evaluation, is the actual atmospheric transmittance, is the atmospheric transmittance evaluated after correction;

[0027] Objective function of terrain correction J topo expression:

[0028] J topo =|B before -B after |

[0029] Among them: B before is the average brightness of the shadow area before correction, B after is the average brightness of the shadow area after correction.

[0030] Furthermore, the multi-objective optimization algorithm is used to iteratively optimize the target values ​​corresponding to each correction task type to obtain a target frontier solution set, including:

[0031] The initial parameters are set as the initial population based on the NSGA-II algorithm, and the population size is determined according to the number of participants and the dimension of the parameters;

[0032] Calculate the target value of each individual in the population and the crowding degree of each individual according to the objective function corresponding to each correction task type;

[0033] Perform selection, crossover, and mutation operations to update initial parameters;

[0034] When the set number of iterations is reached, the target frontier solution set is determined.

[0035] Furthermore, determining the optimal correction scheme according to the target frontier solution set includes:

[0036]

[0037] Where: D(I) is the Euclidean distance of correction scheme I, q is the number of objective functions, J i (I) is the function value of correction scheme I on the i-th correction target, is the optimal value that can be achieved under the i-th correction target.

[0038] Furthermore, each participant performs local model training based on the preprocessed data, extracts initial parameters based on the trained model, encrypts the initial parameters, and sends the encrypted initial parameters to the aggregation end, including:

[0039] Each of the participants determines a model architecture according to the remote sensing image data and the characteristics of the task to be processed, and obtains an initialization model based on the model architecture;

[0040] Optimizing the initialization model according to a set loss function to obtain an optimized local model;

[0041] Extracting initial parameters according to the type of the optimized local model;

[0042] The initial parameters are encrypted using a homomorphic encryption algorithm, and the encrypted initial parameters are sent to the aggregation end through a preset data transmission protocol.

[0043] Furthermore, the aggregation end extracts a target feature vector set according to the corrected remote sensing image data, aggregates the target feature vector set through a Bayesian model, and generates target parameters, including:

[0044] Determine the prior probability and likelihood function based on the local model of each participant using a Bayesian algorithm;

[0045] Calculating a posterior probability based on the prior probability and the likelihood function;

[0046] The target feature vector is weighted averaged and aggregated using the posterior probability to obtain an aggregated feature vector.

[0047] Furthermore, the step of performing weighted average aggregation on the target feature vector using the posterior probability to obtain an aggregated feature vector includes:

[0048]

[0049] in: is the aggregated feature vector of the k′th participant, n′ is the number of participants, P(M i |D) is the posterior probability, is the feature vector of the k′th participant before aggregation.

[0050] A second aspect of an embodiment of the present application provides a remote sensing image processing system based on federated learning, including:

[0051] A remote sensing image data acquisition and processing unit is used for each participant to acquire remote sensing image data through several channels and pre-process the acquired remote sensing image data;

[0052] An initial parameter extraction and encryption unit, which is used for each participant to perform local model training based on the preprocessed data, extract initial parameters based on the trained model, encrypt the initial parameters, and send the encrypted initial parameters to the aggregation end;

[0053] A correction scheme determination unit, configured for the aggregator to decrypt the received ciphertext to obtain the initial parameters, determine an optimal correction scheme based on the initial parameters using a multi-objective optimization algorithm, and correct the remote sensing image data of each participant according to the optimal correction scheme;

[0054] A target parameter generating unit, used for the aggregation end to extract a target feature vector set according to the corrected remote sensing image data, aggregate the target feature vector set through a Bayesian model, and generate target parameters;

[0055] The local model updating unit is used for the aggregation end to distribute the target parameters to the various participants, so that the various participants use the target parameters to update the local model and perform multiple iterations.

[0056] A third aspect of an embodiment of the present application provides a computer device, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, wherein when the processor executes the computer-readable instructions, the steps of the remote sensing image processing method based on federated learning as described in any one of the above are implemented.

[0057] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0058] In the present invention, for model training, each participant performs preprocessing based on the acquired remote sensing image data to provide an accurate data foundation for the local model. The aggregation end weighs the multi-dimensional correction demands through a multi-objective optimization algorithm, and determines the global optimal solution for data correction, which can greatly reduce the errors caused by various correction factors; based on the correction results, the feature vector set is refined, and the Bayesian model is used to aggregate and integrate the advantages of all parties, and the highly reliable target parameters are distributed to the participants. The present invention can comprehensively improve the model performance and data quality, effectively improve the processing efficiency of remote sensing images, improve the accuracy of remote sensing image processing, and can provide users with high-quality images that meet their needs.

[0059] Other advantages, objectives, and features of the present invention will be set forth in part in the following description, and in part will be apparent to those skilled in the art based on an examination of the following or may be taught from the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A schematic diagram of an embodiment of a remote sensing image processing method based on federated learning in the present invention;

[0061] Figure 2 The figure is a flowchart of another embodiment of a remote sensing image processing method based on federated learning in the present invention. DETAILED DESCRIPTION

[0062] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0063] Embodiment 1

[0064] See also Figure 1 , an embodiment of a remote sensing image processing method based on federated learning in the present invention comprises the following steps:

[0065] S11. Each participant obtains remote sensing image data through several channels and pre-processes the obtained remote sensing image data;

[0066] In this embodiment, each participant can obtain remote sensing image data by cooperating with well-known commercial satellite data suppliers based on business priorities and budgets; can obtain remote sensing image data by accessing authoritative free data platforms; can collect data by drones equipped with professional remote sensing equipment; and can share data with local meteorological departments, geographic survey agencies, and agricultural research institutes.

[0067] Data preprocessing includes noise removal, outlier processing, invalid data elimination and data normalization, among which normalization includes band, spatial scale and data format normalization.

[0068] S12. Each participant performs local model training based on the preprocessed data, extracts initial parameters based on the trained model, encrypts the initial parameters, and sends the encrypted initial parameters to the aggregation end;

[0069] In this embodiment, step S12 includes the following:

[0070] 1. Each participant determines the model architecture based on the remote sensing image data and the characteristics of the task to be processed, and obtains the initialization model based on the model architecture;

[0071] Each participant determines the model architecture based on the characteristics of remote sensing image data and the tasks to be processed. For example, if the focus is on accurate classification of land cover, the classic convolutional neural network (CNN) is preferred, such as the ResNet series, which effectively alleviates the gradient vanishing problem with its residual structure and adapts to the training needs of deep networks; if the focus is on monitoring the dynamic changes of land objects over time, the recurrent neural network (RNN) and its long short-term memory network LSTM are more suitable, using internal memory units to capture time series correlation information.

[0072] Initialize the model parameters. The initial setting of weights often follows a specific distribution. Giving the model initial state randomness is conducive to training optimization. The bias item is initially set to 0, as well as key hyperparameters. The local model is initialized based on the initialization parameters.

[0073] 2. Optimize the initialization model according to the set loss function to obtain the optimized local model;

[0074] For example, the tasks to be processed include radiation correction related tasks, geometric correction corresponding tasks, and classification application directional tasks. The radiation correction related tasks aim to minimize the deviation between the corrected radiation value and the theoretical true value. In this case, the mean square error (MSE) loss function is used for optimization. The square operation is used to amplify the error difference and drive the model to accurately fit the real radiation characteristics. For the geometric correction corresponding tasks, the focus is on correcting the geometric distortion of the image. The coordinate matching accuracy is often measured by the reprojection error. The square root of the sum of squared coordinate errors of each control point is accumulated to reflect the accuracy of the geometric correction. For the classification application directional tasks, the purpose is to quantify the difference between the predicted category and the true label. The cross entropy loss function is selected to encourage the model output to be close to the probability distribution of the true label, thereby improving the classification accuracy. The visual task complexity and focus are combined with weighted combination of multiple loss terms for collaborative optimization.

[0075] 3. Extract initial parameters according to the type of optimized local model;

[0076] The local model is trained iteratively for multiple rounds until the model performs stably in the validation set and reaches the preset convergence standard, then the key parameters of the model are extracted. For example, in the CNN model architecture, the extracted initial parameters include the convolution kernel weight matrix and bias vector of each convolution layer, and the weight and bias of the fully connected layer; while the initial parameters extracted from the traditional model generally include core parameters such as polynomial coefficients.

[0077] 4. Use the homomorphic encryption algorithm to encrypt the initial parameters, and send the encrypted initial parameters to the aggregation end through the preset data transmission protocol.

[0078] The homomorphic encryption algorithm here is Paillier encryption, which uses a homomorphic encryption library or related encryption tools to generate public and private keys. After extracting the model parameters, the initial parameters need to be converted into a format suitable for encryption. For example, for the weight matrix and bias vector of the neural network, ensure that their data types and structures meet the input requirements of the encryption algorithm. For each parameter to be encrypted, encrypt it using the generated public key. Taking the convolutional layer weights of a convolutional neural network (CNN) as an example, assuming that the extracted weight matrix is (l is the number of layers), encrypt it with the public key to obtain the encrypted weight matrix For the bias vector and other model parameters, such as the weights W of the fully connected layers fc and bias b fc , and is encrypted in the same way. The encrypted parameters become ciphertext, which can be used for specific homomorphic operations on the aggregation end without being decrypted.

[0079] S13. The aggregator decrypts the received ciphertext to obtain initial parameters, determines the optimal correction scheme based on the initial parameters using a multi-objective optimization algorithm, and corrects the remote sensing image data of each participant according to the optimal correction scheme;

[0080] In this embodiment, after receiving the encrypted ciphertext sent by each participant, the aggregator obtains the corresponding decryption tool or key according to the encryption technology adopted, and uses the decryption tool and key to decrypt the received ciphertext. Then, the NSGA-II algorithm is used to select the optimal correction scheme, which can make all correction targets as close to the optimal situation as possible, and then the remote sensing image data of each participant is corrected.

[0081] S14. The aggregation end extracts a target feature vector set according to the corrected remote sensing image data, aggregates the target feature vector set through a Bayesian model, and generates target parameters;

[0082] Step S14 includes the following steps:

[0083] 1. Use the Bayesian algorithm to determine the prior probability and likelihood function based on the local model of each participant;

[0084] For each participant’s model, assuming there are n′ participants, we need to determine the prior probability P(M i ), P(M i ) reflects the degree of trust in each model before seeing the data. The determination of prior probability can be based on factors such as the quality of the participant's data, the amount of data, the complexity of the model, or historical performance. For example, if a participant's data has been rigorously preprocessed and has performed well in similar tasks before, then the participant's model can be given a higher prior probability.

[0085] The likelihood function can be expressed as P(D|M i ), that is, given the model M i The probability of observing data D under the condition of . For the feature vector set after correction of remote sensing image data, the likelihood function can be defined based on the degree of fit of the model to the data. For example, in a classification task, if the model predicts that the probability of a pixel belonging to a certain category is closer to the category to which the pixel actually belongs, the value of the likelihood function will be higher.

[0086] 2. Calculate the posterior probability based on the prior probability and likelihood function;

[0087] According to Bayes' theorem, the posterior probability P(M) of each model is calculated. i |D), as follows:

[0088]

[0089] The posterior probability combines the prior probability and the likelihood function, reflecting the updated confidence in each model after seeing the data.

[0090] 3. Use the posterior probability to perform weighted average aggregation on the target feature vector to obtain the aggregated feature vector.

[0091] For a given image data feature or prediction task, assume that each participant model M i The output feature vector in It can represent vectors containing radiation features, geometric features, etc., and use the posterior probability of each model to perform weighted average aggregation on the feature vectors of the model. The aggregated feature vector The expression is as follows:

[0092]

[0093] Where: n′ is the number of participants.

[0094] For example, in the aggregation of radiation feature elements, the radiation features of each participant model weighted based on the posterior probability are added together to obtain the aggregated radiation features, and the aggregation of the entire feature vector is completed in this way, and the target parameters are determined by the aggregated feature vector.

[0095] For example, if the task is to predict the land cover classification, each pixel belongs to the category, each model M i The probability of predicting a pixel belonging to category C is P(C|M i ), then the probability P(C) that the aggregated pixel belongs to category C is: In this way, the prediction results of the models of each participant are weighted and averaged to obtain a comprehensive prediction result that takes into account the credibility of each model.

[0096] S15. The aggregation end distributes the target parameters to each participant, so that each participant uses the target parameters to update the local model and perform multiple iterations.

[0097] After the aggregator generates the target parameters by means of Bayesian model average aggregation and other methods, it sends the target parameters to the corresponding participants according to the pre-set participant identification or communication address and other information. When the participant receives the target parameters from the aggregator, it updates the local model. If the participant uses a neural network-based model, the weights and bias parameters in the newly received target parameters will replace the original weights and bias parameters in the local model. Each participant uses the updated local model and the prepared remote sensing image data to start a new round of iterative training. During the training process, the model will learn and adjust its internal representation again based on the new parameters and data features, and continuously optimize the processing capabilities of remote sensing image data. Through repeated iterative training processes, the model can continuously absorb the optimization results of the aggregator and perform in-depth optimization based on the characteristics of local data. With the increase in the number of iterations, the accuracy, stability and generalization ability of remote sensing image correction can be improved.

[0098] Embodiment 2

[0099] See also Figure 2 In the present invention, the aggregator decrypts the received ciphertext to obtain initial parameters, determines the optimal correction scheme based on the initial parameters using a multi-objective optimization algorithm, and corrects the remote sensing image data of each participant according to the optimal correction scheme, and also includes the following steps:

[0100] S21. The aggregation end stores the initial parameters in a preset data structure based on the correction task type of the initial parameters;

[0101] After receiving and decrypting the model parameters of the participants, the aggregation end must first organize and classify these parameters. Specifically, the parameters are stored in different sets or data structures according to the correction type to which the parameters belong. It should be noted that the present invention limits the correction types to four types: radiation correction, geometric correction, atmospheric correction, and terrain correction, and performs the following calculations without considering other correction types. For example, if it is a correction model based on a neural network, the convolutional layer weights and bias parameters related to radiation correction in the neural network are placed in one set, and the parameters related to geometric correction are placed in another set. At the same time, the relevant information of each participant is recorded, such as data quality indicators, data volume, data source area and other information, which will be used for subsequent weight calculations and solution evaluation.

[0102] S22. Determine the corresponding objective function based on each correction task type;

[0103] The four correction task types defined above include radiation correction, geometric correction, atmospheric correction, and terrain correction.

[0104] Objective function J of radiation correction rad expression:

[0105]

[0106] Where: n is the number of samples, is the corrected radiation value, y i is the actual radiation value;

[0107] Objective function J of geometric correction geo expression:

[0108]

[0109] Where: m is the number of ground control points, (X j , Y j ) is the actual geographic coordinate, (x j ,y j ) are the image coordinates after geometric correction;

[0110] Objective function of atmospheric correction J atm expression:

[0111]

[0112] Where: k is the number of sample points for evaluation, is the actual atmospheric transmittance, is the atmospheric transmittance evaluated after correction;

[0113] Objective function of terrain correction J topo expression:

[0114] J topo =|B before -B after |

[0115] Among them: B before is the average brightness of the shadow area before correction, B after is the average brightness of the shadow area after correction.

[0116] S23. Iteratively optimize the target values ​​corresponding to each correction task type using a multi-objective optimization algorithm to obtain a target frontier solution set;

[0117] Step S23 also includes the following steps:

[0118] 1. Based on the NSGA-II algorithm, the initial parameters are set as the initial population, and the population size is determined according to the number of participants and the dimension of the parameters;

[0119] The sorted model parameters of the participants are used as the initial population. Each individual represents a possible correction scheme combination, and the genes of the individual include the parameters of each correction task. For example, the parameters of an individual may include the atmospheric optical thickness estimate in radiation correction, the coefficients of the geometric correction polynomial, the aerosol model parameters in atmospheric correction, the sun-ground-sensor geometric relationship parameters in terrain correction, etc.

[0120] The population size is determined based on the number of participants and the dimension of the parameters. Assuming there are p participants and the dimension of the model parameters of each participant is d, the population size can be set to N = m × p × d, where m is a coefficient greater than 1.

[0121] 2. Calculate the target value of each individual in the population and the crowding degree of each individual according to the objective function corresponding to each correction task type;

[0122] For each individual in the population, the objective function determined by the above steps, namely J rad , J geo , J atm and J topo Calculate the corresponding target value. Then, sort the individuals according to the Pareto dominance relationship. An individual A dominates another individual B if and only if A's value on all objective functions is not inferior to B, and its value on at least one objective function is better than B. In this way, the population is divided into different non-dominated layers. The first layer is the non-inferior solution set, that is, the Pareto frontier, where these individuals achieve a good balance between all objectives.

[0123] In order to maintain the diversity of the population, in addition to non-dominated sorting, it is also necessary to calculate the crowding of individuals. For each objective function, the individuals are sorted according to the value of the objective function. The distance between each individual and its adjacent individuals is calculated, and the sum of these distances is taken as the crowding of the individual.

[0124] 3. Perform selection, crossover, and mutation operations to update initial parameters;

[0125] Selection operation: The parent individuals are selected from the population using the tournament selection method. For example, in the tournament selection, t individuals are randomly selected, and then the individual with the highest non-dominated ranking is selected from these t individuals as the parent.

[0126] Crossover operation: Perform a crossover operation on the selected parent individuals to generate new offspring individuals. The crossover method uses simulated binary crossover (SBX). Assuming two parent individuals P1 and P2, for a certain parameter x, the parameter values ​​of the offspring individuals C1 and C2 after the crossover are calculated as follows:

[0127]

[0128]

[0129] Where: β is a random number controlled by the cross-distribution index, which is usually determined based on the characteristics of the problem and experience.

[0130] Mutation operation: Perform mutation operation on the newly generated offspring individuals to introduce new gene combinations and diversity. For real number parameters, mutation can be achieved by adding a random perturbation to the parameter value.

[0131] 4. When the set number of iterations is reached, the target frontier solution set is determined.

[0132] Repeat the process of non-dominated sorting, crowding calculation and genetic operation for multiple rounds of iterations. After each iteration, determine whether the population has converged. The convergence condition can be that the maximum number of iterations is reached, or the moving distance of the Pareto frontier of the population in the objective function space for several consecutive generations is less than a certain threshold.

[0133] S24. Determine the optimal correction scheme based on the target frontier solution set.

[0134] When the algorithm converges, the final Pareto front solution set is obtained. This solution set contains correction schemes that achieve different balances between multiple correction targets. According to actual needs and application scenarios, the optimal correction scheme is selected from the Pareto frontier. If each correction target is given equal importance, the individual closest to the ideal point (the point where all objective function values ​​are the smallest) in the objective function space can be selected as the optimal solution. Here, the Euclidean distance is used to represent the optimal solution, and the expression is as follows:

[0135]

[0136] Where: D(I) is the Euclidean distance of correction scheme I, q is the number of objective functions, J i (I) is the function value of correction scheme I on the i-th correction target, is the optimal value that can be achieved under the i-th correction target.

[0137] If a correction target is more emphasized in a certain application scenario, for example, the geometric correction effect is more emphasized in high-precision map production, the individual with the smallest geometric correction objective function value can be selected as the optimal solution in the Pareto frontier, or a higher weight can be given to the geometric correction objective function to adjust the distance calculation method.

[0138] Based on the above, the optimal correction scheme is selected and its corresponding correction parameters are applied to the remote sensing image data of all participants. For example, if the radiation correction parameters in the optimal scheme include new atmospheric optical thickness estimates and coefficient adjustments in the radiation transfer model, all images are radiometrically corrected according to these parameters.

[0139] For geometric correction, the selected geometric transformation model (such as polynomial geometric correction model) and the corresponding coefficients are used to correct the geometric deformation of the image. In a similar way, atmospheric correction and terrain correction parameters are applied to complete the unified correction of all remote sensing image data.

[0140] In the above-mentioned embodiment, after the data of multiple participants are corrected, these data are more matched in the feature space, which is conducive to data fusion and collaborative training at the aggregation end. By selecting a suitable correction scheme through a multi-objective optimization algorithm, the characteristics and advantages of the data of each participant can be fully considered, and data from different sources can be integrated together, so that the model can learn more comprehensive and representative features, thereby improving the generalization ability and performance of the federated learning model.

[0141] Embodiment 3

[0142] An embodiment of a remote sensing image processing system based on federated learning in the present invention comprises the following steps:

[0143] A remote sensing image data acquisition and processing unit is used for each participant to acquire remote sensing image data through several channels and pre-process the acquired remote sensing image data;

[0144] An initial parameter extraction and encryption unit, which is used for each participant to perform local model training based on the preprocessed data, extract initial parameters based on the trained model, encrypt the initial parameters, and send the encrypted initial parameters to the aggregation end;

[0145] A correction scheme determination unit, configured for the aggregator to decrypt the received ciphertext to obtain the initial parameters, determine an optimal correction scheme based on the initial parameters using a multi-objective optimization algorithm, and correct the remote sensing image data of each participant according to the optimal correction scheme;

[0146] A target parameter generating unit, used for the aggregation end to extract a target feature vector set according to the corrected remote sensing image data, aggregate the target feature vector set through a Bayesian model, and generate target parameters;

[0147] The local model updating unit is used for the aggregation end to distribute the target parameters to the various participants, so that the various participants use the target parameters to update the local model and perform multiple iterations.

[0148] Embodiment 4

[0149] The present invention provides a computer device, comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, the steps of the above method are implemented.

[0150] Those of ordinary skill in the art will appreciate that the units of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition of each example has been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0151] In the embodiments provided by the present invention, it should be understood that the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units can be combined into one unit, one unit can be split into multiple units, or some features can be ignored. In addition, each functional unit in each embodiment of the present invention can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0152] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-0nlyMemory), random access memory (RAM, RandomAccessMemory), mobile hard disk, magnetic disk or optical disk, etc., which can store program code.

[0153] It can be understood that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A remote sensing image processing method based on federated learning, characterized in that: include: Each participant obtains remote sensing image data through several channels and pre-processes the obtained remote sensing image data; Each participant performs local model training based on the preprocessed data, extracts initial parameters based on the trained model, encrypts the initial parameters, and sends the encrypted initial parameters to the aggregation end; The aggregator decrypts the received ciphertext to obtain the initial parameters, determines an optimal correction scheme based on the initial parameters using a multi-objective optimization algorithm, and corrects the remote sensing image data of each participant according to the optimal correction scheme; The aggregation end extracts a target feature vector set according to the corrected remote sensing image data, aggregates the target feature vector set through a Bayesian model, and generates target parameters; The aggregation end distributes the target parameters to the various participants, so that the various participants use the target parameters to update the local model and perform multiple iteration operations.

2. The remote sensing image processing method based on federated learning according to claim 1, characterized in that: The aggregator decrypts the received ciphertext to obtain the initial parameters, determines an optimal correction scheme based on the initial parameters using a multi-objective optimization algorithm, and corrects the remote sensing image data of each participant according to the optimal correction scheme, including: The aggregation end stores the initial parameters in a preset data structure respectively based on the correction task type of the initial parameters; Determine the corresponding objective function based on each correction task type; The multi-objective optimization algorithm is used to iteratively optimize the target values ​​corresponding to each correction task type to obtain the target frontier solution set; An optimal correction scheme is determined according to the target frontier solution set.

3. The remote sensing image processing method based on federated learning according to claim 2, characterized in that: The determining of the corresponding objective function based on each correction task type includes: The various correction task types include radiation correction, geometric correction, atmospheric correction and terrain correction; Objective function J of radiation correction rad expression: Where: n is the number of samples, is the corrected radiation value, y i is the actual radiation value; Objective function J of geometric correction geo expression: Where: m is the number of ground control points, (X j ,Y j ) is the actual geographic coordinate, (x j ,y j ) are the image coordinates after geometric correction; Objective function of atmospheric correction J atm expression: Where: k is the number of sample points for evaluation, is the actual atmospheric transmittance, is the atmospheric transmittance evaluated after correction; Objective function of terrain correction J topo expression: J topo =|B before -B after | Among them: B before is the average brightness of the shadow area before correction, B after is the average brightness of the shadow area after correction.

4. The remote sensing image processing method based on federated learning according to claim 3, characterized in that: The multi-objective optimization algorithm is used to iteratively optimize the target values ​​corresponding to each correction task type to obtain a target frontier solution set, including: The initial parameters are set as the initial population based on the NSGA-II algorithm, and the population size is determined according to the number of participants and the dimension of the parameters; Calculate the target value of each individual in the population and the crowding degree of each individual according to the objective function corresponding to each correction task type; Perform selection, crossover, and mutation operations to update initial parameters; When the set number of iterations is reached, the target frontier solution set is determined.

5. The remote sensing image processing method based on federated learning according to any one of claims 1 to 4, characterized in that: Determining the optimal correction scheme according to the target frontier solution set includes: Where: D(I) is the Euclidean distance of correction scheme I, q is the number of objective functions, J i (I) is the function value of correction scheme I on the i-th correction target, is the optimal value that can be achieved under the i-th correction target.

6. The remote sensing image processing method based on federated learning according to claim 1, characterized in that: Each participant performs local model training based on the preprocessed data, extracts initial parameters based on the trained model, encrypts the initial parameters, and sends the encrypted initial parameters to the aggregation end, including: Each of the participants determines a model architecture according to the remote sensing image data and the characteristics of the task to be processed, and obtains an initialization model based on the model architecture; Optimizing the initialization model according to a set loss function to obtain an optimized local model; Extracting initial parameters according to the type of the optimized local model; The initial parameters are encrypted using a homomorphic encryption algorithm, and the encrypted initial parameters are sent to the aggregation end through a preset data transmission protocol.

7. The remote sensing image processing method based on federated learning according to claim 1, characterized in that: The aggregation end extracts a target feature vector set according to the corrected remote sensing image data, aggregates the target feature vector set through a Bayesian model, and generates target parameters, including: Determine the prior probability and likelihood function based on the local model of each participant using a Bayesian algorithm; Calculating a posterior probability based on the prior probability and the likelihood function; The target feature vector is weighted averaged and aggregated using the posterior probability to obtain an aggregated feature vector.

8. The remote sensing image processing method based on federated learning according to claim 7, characterized in that: The step of performing weighted average aggregation on the target feature vector by using the posterior probability to obtain an aggregated feature vector includes: in: is the aggregated feature vector of the k′th participant, n′ is the number of participants, P(M i |D) is the posterior probability, is the feature vector of the k′th participant before aggregation.

9. A remote sensing image processing system based on federated learning, characterized in that: include: A remote sensing image data acquisition and processing unit is used for each participant to acquire remote sensing image data through several channels and pre-process the acquired remote sensing image data; An initial parameter extraction and encryption unit, which is used for each participant to perform local model training based on the preprocessed data, extract initial parameters based on the trained model, encrypt the initial parameters, and send the encrypted initial parameters to the aggregation end; A correction scheme determination unit, configured for the aggregator to decrypt the received ciphertext to obtain the initial parameters, determine an optimal correction scheme based on the initial parameters using a multi-objective optimization algorithm, and correct the remote sensing image data of each participant according to the optimal correction scheme; A target parameter generating unit, used for the aggregation end to extract a target feature vector set according to the corrected remote sensing image data, aggregate the target feature vector set through a Bayesian model, and generate target parameters; The local model updating unit is used for the aggregation end to distribute the target parameters to the various participants, so that the various participants use the target parameters to update the local model and perform multiple iterations.

10. A computer device comprising a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor, characterized in that: When the processor executes the computer-readable instructions, the steps of the remote sensing image processing method based on federated learning as described in any one of claims 1 to 8 are implemented.

Images