Prediction method, device, storage medium and remote sensing equipment based on federated learning

Abstract

The present invention discloses a prediction method, device, storage medium and remote sensing equipment based on federated learning, the method comprising: training the initial model according to local training data to obtain a unilateral prediction model; transmitting the unilateral prediction model to a ground information center, the ground information center aggregating several unilateral prediction models corresponding to several remote sensing devices to obtain a federated prediction model, the federated prediction model comprising several federated sub-models; receiving the corresponding federated sub-models, and predicting the local data to be predicted based on the federated sub-models to obtain a prediction label. Thus, by training the model locally on the remote sensing device and realizing the comprehensive utilization of high-altitude data based on federated learning, the technical bottleneck of data transmission is broken.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a prediction method, device, storage medium and remote sensing equipment based on federated learning. Background Art

[0002] With the development of computer technology, more and more technologies (big data, distributed, blockchain, artificial intelligence, etc.) are being applied in the financial field. The traditional financial industry is gradually transforming to financial technology (Fintech). However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology.

[0003] Currently, more and more remote sensing equipment such as artificial satellites are operating in the sky, performing their respective tasks and obtaining corresponding data. As we all know, remote sensing technology is difficult and expensive, and the data obtained by remote sensing equipment is very precious. If the data of various remote sensing equipment are to be comprehensively utilized, these data need to be aggregated to the ground information center, which will perform data analysis and processing. However, there is a network delay between the remote sensing equipment and the ground information center, and the amount of high-altitude data obtained by the remote sensing equipment is huge, which requires high network bandwidth, and thus data transmission has become a technical bottleneck. Summary of the invention

[0004] The present invention provides a prediction method, device, storage medium and remote sensing equipment based on federated learning, aiming to realize the comprehensive utilization of high-altitude data and break the technical bottleneck of data transmission.

[0005] To achieve the above object, the present invention provides a prediction method based on federated learning, which is applied to a remote sensing system and includes:

[0006] The initial model is trained according to the local training data to obtain a unilateral prediction model;

[0007] The unilateral prediction model is transmitted to a ground information center, and the ground information center aggregates a plurality of the unilateral prediction models corresponding to a plurality of remote sensing devices to obtain a federated prediction model, wherein the federated prediction model includes a plurality of federated sub-models;

[0008] The corresponding federated sub-model is received, and the local data to be predicted is predicted based on the federated sub-model to obtain a prediction label.

[0009] Preferably, the step of training the initial model according to the local training data to obtain the unilateral prediction model comprises:

[0010] Randomly obtain initial model parameters, use the initial model parameters to obtain initial predicted labels of the local training data, and calculate a loss function based on the initial predicted labels and actual labels of the local training data;

[0011] Based on the loss function, updating the model parameters by gradient descent;

[0012] If the convergence condition is reached, the update is stopped and the corresponding model parameters are saved as the final model parameters to obtain a unilateral prediction model.

[0013] Preferably, the plurality of remote sensing devices have different inclination angles, and the step of predicting the local data to be predicted based on the federated sub-model and obtaining the prediction label further includes:

[0014] Determining whether the local data to be predicted of each of the remote sensing devices includes a plurality of target data of the same dimension;

[0015] If the local data to be predicted of each of the remote sensing devices includes a plurality of target data of the same dimension, the following steps are executed: predicting the local data to be predicted based on the federated sub-model to obtain a prediction label.

[0016] Preferably, before the step of training the initial model according to the local training data to obtain the unilateral prediction model, the step further includes:

[0017] Receiving instructions sent by the ground information center through a control system, executing tasks according to the instructions, and obtaining the local data, wherein the local data includes image data;

[0018] The local data is preprocessed to obtain local training data.

[0019] Preferably, the step of determining whether the local data to be predicted of each remote sensing device includes a plurality of target data of the same dimension comprises:

[0020] Determine several dimensions to be judged based on the prediction task;

[0021] Acquire local dimensions in the local data to be predicted of each remote sensing device, and determine whether the local dimensions include a plurality of identical dimensions based on the dimensions to be determined;

[0022] If the local dimensions all include the several dimensions to be determined, then determining that the local dimensions of each remote sensing device include several identical dimensions;

[0023] If the local dimensions of each of the remote sensing devices include a plurality of identical dimensions, searching the local data to be predicted for target data corresponding to the plurality of dimensions to be determined;

[0024] If target data corresponding to the several dimensions to be determined exist in the local data of each remote sensing device, it is determined that the local data to be predicted of each remote sensing device includes several target data of the same dimensions.

[0025] Preferably, the step of preprocessing the local data to obtain local training data includes:

[0026] The local data is preprocessed according to the process, and the preprocessed local data is saved as the local training data.

[0027] Preferably, the step of receiving the corresponding federated sub-model, predicting the local data to be predicted based on the federated sub-model, and obtaining the prediction label includes:

[0028] Receiving the federation sub-model returned by the ground information center;

[0029] The local data to be predicted is input into the federated sub-model, and the federated sub-model outputs the corresponding prediction label.

[0030] In addition, to achieve the above-mentioned purpose, the present invention further provides a prediction device based on federated learning, and the prediction device based on federated learning includes:

[0031] A training module, used to train the initial model according to local training data to obtain a unilateral prediction model;

[0032] A transmission module, used for transmitting the unilateral prediction model to a ground information center, and the ground information center aggregates a plurality of the unilateral prediction models corresponding to a plurality of remote sensing devices to obtain a federated prediction model, wherein the federated prediction model includes a plurality of federated sub-models;

[0033] The prediction module is used to receive the corresponding federated sub-model, and predict the local data to be predicted based on the federated sub-model to obtain a prediction label.

[0034] In addition, to achieve the above-mentioned purpose, the present invention also provides a remote sensing device, which includes a processor, a memory, and a prediction program based on federated learning stored in the memory. When the prediction program based on federated learning is executed by the processor, the steps of the prediction method based on federated learning as described above are implemented.

[0035] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer storage medium, on which a prediction program based on federated learning is stored. When the prediction program based on federated learning is executed by a processor, the steps of the prediction method based on federated learning as described above are implemented.

[0036] Compared with the prior art, the present invention discloses a prediction method, device, storage medium and remote sensing equipment based on federated learning, which trains the initial model according to local training data to obtain a unilateral prediction model; transmits the unilateral prediction model to the ground information center, and the ground information center aggregates several unilateral prediction models corresponding to several remote sensing devices to obtain a federated prediction model, which includes several federated sub-models; receives the corresponding federated sub-models, and predicts the local data to be predicted based on the federated sub-models to obtain a prediction label. Therefore, by training the model locally on the remote sensing device and realizing the comprehensive utilization of high-altitude data based on federated learning, the technical bottleneck of data transmission is broken. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a schematic diagram of the hardware structure of the remote sensing device involved in each embodiment of the present invention;

[0038] Figure 2 It is a flowchart of the first embodiment of the prediction method based on federated learning of the present invention;

[0039] Figure 3 It is a schematic diagram of a scenario of an embodiment of a prediction method based on federated learning of the present invention;

[0040] Figure 4 is a flow chart of a second embodiment of a prediction method based on federated learning of the present invention;

[0041] Figure 5 It is a functional module diagram of the first embodiment of the prediction device based on federated learning of the present invention.

[0042] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0043] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0044] The remote sensing device mainly involved in the embodiments of the present invention refers to a network connection device that can achieve network connection. The remote sensing device can be a server, a cloud platform, etc.

[0045] Reference Figure 1 , Figure 11 is a schematic diagram of the hardware structure of the remote sensing device involved in each embodiment of the present invention. In the embodiment of the present invention, the remote sensing device may include a processor 1001 (such as a central processing unit, CPU), a communication bus 1002, an input port 1003, an output port 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components; the input port 1003 is used for data input; the output port 1004 is used for data output, and the memory 1005 can be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 can optionally be a storage device independent of the aforementioned processor 1001. Those skilled in the art can understand that Figure 1 The hardware structure shown in the figure does not constitute a limitation of the present invention, and may include more or less components than those shown in the figure, or combine certain components, or arrange the components differently.

[0046] Continue to refer to Figure 1 , Figure 1 The memory 1005 as a readable storage medium may include an operating system, a network communication module, an application module, and a prediction program based on federated learning. Figure 1 In the example, the network communication module is mainly used to connect to the server and perform data communication with the server; and the processor 1001 can call the prediction program based on federated learning stored in the memory 1005 and perform the following operations:

[0047] The initial model is trained according to the local training data to obtain a unilateral prediction model;

[0048] The unilateral prediction model is transmitted to a ground information center, and the ground information center aggregates a plurality of the unilateral prediction models corresponding to a plurality of remote sensing devices to obtain a federated prediction model, wherein the federated prediction model includes a plurality of federated sub-models;

[0049] The corresponding federated sub-model is received, and the local data to be predicted is predicted based on the federated sub-model to obtain a prediction label.

[0050] Furthermore, the processor 1001 may also be used to call a prediction program based on federated learning stored in the memory 1005, and perform the following steps:

[0051] Randomly obtain initial model parameters, use the initial model parameters to obtain initial predicted labels of the local training data, and calculate a loss function based on the initial predicted labels and actual labels of the local training data;

[0052] Based on the loss function, the model parameters are updated by gradient descent; if the convergence condition is reached, the update is stopped and the corresponding model parameters are saved as the final model parameters, so as to obtain a unilateral prediction model.

[0053] Furthermore, the processor 1001 may also be used to call a prediction program based on federated learning stored in the memory 1005, and perform the following steps:

[0054] Determining whether the local data to be predicted of each of the remote sensing devices includes a plurality of target data of the same dimension;

[0055] If the local data to be predicted of each of the remote sensing devices includes a plurality of target data of the same dimension, the following steps are executed: predicting the local data to be predicted based on the federated sub-model to obtain a prediction label.

[0056] Furthermore, the processor 1001 may also be used to call a prediction program based on federated learning stored in the memory 1005, and perform the following steps:

[0057] Receiving instructions sent by the ground information center through a control system, executing tasks according to the instructions, and obtaining the local data, wherein the local data includes image data;

[0058] The local data is preprocessed to obtain local training data.

[0059] Furthermore, the processor 1001 may also be used to call a prediction program based on federated learning stored in the memory 1005, and perform the following steps:

[0060] Determine several dimensions to be judged based on the prediction task;

[0061] Acquire local dimensions in the local data to be predicted of each remote sensing device, and determine whether the local dimensions include a plurality of identical dimensions based on the dimensions to be determined;

[0062] If the local dimensions all include the several dimensions to be determined, then determining that the local dimensions of each remote sensing device include several identical dimensions;

[0063] If the local dimensions of each of the remote sensing devices include a plurality of identical dimensions, searching the local data to be predicted for target data corresponding to the plurality of dimensions to be determined;

[0064] If target data corresponding to the several dimensions to be determined exist in the local data of each remote sensing device, it is determined that the local data to be predicted of each remote sensing device includes several target data of the same dimensions.

[0065] Furthermore, the processor 1001 may also be used to call a prediction program based on federated learning stored in the memory 1005, and perform the following steps:

[0066] The local data is preprocessed according to the process, and the preprocessed local data is saved as the local training data.

[0067] Furthermore, the processor 1001 may also be used to call a prediction program based on federated learning stored in the memory 1005, and perform the following steps:

[0068] Receiving the federation sub-model returned by the ground information center;

[0069] The local data to be predicted is input into the federated sub-model, and the federated sub-model outputs the corresponding prediction label.

[0070] Based on the above structure, various embodiments of the prediction method based on federated learning of the present invention are proposed.

[0071] Reference Figure 2 , Figure 2 It is a flowchart of the first embodiment of the prediction method based on federated learning of the present invention.

[0072] In this embodiment, the prediction method based on federated learning is applied to remote sensing equipment, and the method includes:

[0073] Step S101: training the initial model according to local training data to obtain a unilateral prediction model;

[0074] In this embodiment, the remote sensing equipment generally refers to high-altitude equipment that uses remote sensing technology to work, such as towers, balloons, airplanes, rockets, artificial satellites, spacecraft and space shuttles. Remote sensing technology refers to a comprehensive technology that uses remote sensing equipment far away from the ground to detect electromagnetic wave radiation information on the earth's surface through sensors, and then transmits, processes and interprets information to detect and monitor the earth's resources and environment. Detection is carried out from a long distance in the form of a high-altitude bird's-eye view, including remote sensing images of multiple points, multiple spectrums, multiple time periods and multiple heights, and remote sensing information that has been enhanced multiple times. It can provide comprehensive systematic, instantaneous or synchronous continuous regional synchronous information, and has great advantages in the application of environmental science. Remote sensing equipment has many uses, such as terrain detection, oil detection, atmospheric environment, and terrain detection, etc.

[0075] Usually each remote sensing device has its own inclination angle (the angle between the orbital plane of the remote sensing device and the equatorial plane of the earth is called the orbital inclination). Therefore, the images of these remote sensing devices are all taken at a certain angle. In this embodiment, the image data is marked as local data.

[0076] In this embodiment, the step S101: training the initial model according to the local training data to obtain the unilateral prediction model also includes:

[0077] Step S1001: receiving the instruction sent by the ground information center through the control system, executing the task according to the instruction, and obtaining the local data, wherein the local data includes image data;

[0078] Generally, the remote sensing device needs to establish a communication connection with a ground information center, so that the remote sensing device can perform data transmission with the ground information center.

[0079] The remote sensing device is pre-equipped with a control system, which includes multiple preset instructions and can also receive control instructions sent by a ground information center, and perform corresponding tasks according to the preset instructions and / or the control instructions. The tasks can be map drawing, terrain survey, etc.

[0080] When the remote sensing device receives the instruction sent by the ground information center through the control system, it executes the task according to the instruction and obtains the corresponding local data. Generally, the local data is image data.

[0081] Step S1002: pre-process the local data to obtain local training data.

[0082] After obtaining the local data, the local data is preprocessed. Specifically, the step of preprocessing the local data to obtain local training data includes:

[0083] The local data is preprocessed according to the process, and the preprocessed local data is saved as the local training data.

[0084] The local initial data is preprocessed according to the process to obtain local training data that can be used for model training. The preprocessing includes noise reduction, geometric correction, image enhancement and image cropping, etc., and the preprocessed local data is saved as the local training data. The local initial data can be image data. Generally, the local training data can be subjected to noise reduction processing by a filter. In the remote sensing imaging process, due to the combined influence of various factors, the geometric position, shape, size, dimension, orientation and other characteristics of the objects on the original image are often inconsistent with the characteristics of the corresponding ground objects. This inconsistency is geometric deformation, also known as geometric distortion, and therefore requires correction processing. The process refers to the order of preprocessing, which is generally set according to experience, for example, according to the following process: noise reduction, geometric correction, image cropping.

[0085] In this embodiment, the local training data is obtained after preprocessing the local data, machine learning training is performed on the randomly obtained model parameters based on the local training data, the model parameters are iteratively updated continuously until convergence, and the final model parameters are saved to obtain a unilateral prediction model.

[0086] In this embodiment, several remote sensing devices are selected, and the several remote sensing devices have different inclination angles, and a panoramic remote sensing system is formed based on the inclination angles. For example, a 360° panoramic remote sensing system can also be a 180° remote sensing system or a 270° remote sensing system. For a 360° panoramic remote sensing system, several remote sensing devices with different inclination angles must be selected, and the selected remote sensing devices can obtain 360° panoramic data.

[0087] Specifically, refer to Figure 3 , Figure 3 Schematic diagram of a scenario of an embodiment of a prediction method based on federated learning of the present invention. Figure 3 In a, six remote sensing devices with different inclination angles are selected. The inclination angles of these six remote sensing devices are θ1, θ2, θ3, θ4, θ5, and θ6. These six remote sensing devices form a 360° panoramic remote sensing system. Figure 3 In b, four remote sensing devices C1, C2, C3 and C4 are selected, among which C1, C2, C3 and C4 have different inclination angles, and data of different angles and directions can be obtained, thus forming a remote sensing system, and the remote sensing system maintains communication connection with the ground information center S.

[0088] In this embodiment, the plurality of remote sensing devices perform model training respectively to obtain a plurality of unilateral prediction models, and the unilateral prediction models are expressed as: Where M represents the model, i represents the remote sensing device number, and t represents the current iteration number. If the current model iteration number is t, the initial model is trained based on the local training data. Training is performed until the initial model Convergence, then we get

[0089] Step S102: transmitting the unilateral prediction model to a ground information center, and the ground information center aggregates a plurality of the unilateral prediction models corresponding to a plurality of remote sensing devices to obtain a federated prediction model, wherein the federated prediction model includes a plurality of federated sub-models;

[0090] After the unilateral prediction model is obtained, the unilateral prediction model is transmitted to the ground information center through the connection established in advance with the ground information center. The ground information center receives several unilateral prediction models uploaded by several remote sensing devices, aggregates the several unilateral prediction models, and obtains a federated prediction model.

[0091] The ground information center aggregates multiple unilateral prediction models to obtain a federated prediction model M t+1 :

[0092]

[0093] Furthermore, the federation model is expressed as And send the federated sub-model to the plurality of remote sensing devices.

[0094] Step S103: receiving the corresponding federated sub-model, and predicting the local data to be predicted based on the federated sub-model to obtain a prediction label.

[0095] The remote sensing device receives the corresponding federated sub-model returned by the ground information center, and predicts the local data to be predicted based on the federated sub-model to obtain a prediction label.

[0096] Specifically, the step of receiving the corresponding federated sub-model, predicting the local data to be predicted based on the federated sub-model, and obtaining the prediction label includes:

[0097] Step S103a: receiving the federation sub-model returned by the ground information center;

[0098] Receive the federated sub-model returned by the ground information center. Save the federated sub-model to a preset storage path, and update the federated sub-model of the remote sensing device. Specifically, the federated sub-model can be saved directly. If the remote sensing device already has an original federated sub-model, the original federated sub-model is updated, and the saved federated sub-model and the original federated sub-model can also be compared to obtain different difference parameters, and the original federated sub-model is updated based on the difference. Generally, after receiving the federated sub-model returned by the ground information center, the federated sub-model is marked as the latest model, and the latest model is used for prediction. In other embodiments, the original federated sub-model can be deleted or deactivated.

[0099] Step S103b: input the local data to be predicted into the federated sub-model, and the federated sub-model outputs the corresponding prediction label.

[0100] In this embodiment, the local data to be predicted is input into the federal sub-model, and the federal sub-model processes the local data to be predicted according to the model parameters to obtain a prediction label. The prediction label can be 0, 1, and others. The prediction label can also include prediction results and probabilities. It should be noted that the local data obtained by the remote sensing equipment according to the task is pre-processed in advance to obtain the local data to be predicted, and the pre-processing process is consistent with the above-mentioned process of processing the local initial data to obtain local training data. The federal sub-model predicts the local data to be predicted and outputs a prediction label, and the prediction label is related to the specific task. For example, for terrain survey, the prediction label can be the terrain type of the local data to be predicted, and the probability of belonging to this type. For another example, for oil survey, the prediction label can be yes or no, which can be represented by the corresponding numbers 1 and 0.

[0101] The step S103: predicting the local data to be predicted based on the federated sub-model, and obtaining the prediction label before the step further includes:

[0102] Step S103-1: determining whether the local data to be predicted of each remote sensing device includes a plurality of target data of the same dimension;

[0103] In this embodiment, the local data acquired by the remote sensing devices have several same dimensions. Therefore, in order to obtain the data to be predicted that meets the requirements, it can be determined in advance whether the local data to be predicted of each remote sensing device includes target data of the same dimension.

[0104] Specifically, the step S103-1: determining whether the local data to be predicted of each remote sensing device includes a plurality of target data of the same dimension comprises:

[0105] Step S103-1a: determining a number of dimensions to be determined based on the prediction task;

[0106] It is understandable that the local data to be predicted includes data of multiple dimensions. Based on different tasks, the local data obtained by remote sensing equipment of different tasks includes one or more data of different dimensions, and the local data obtained by remote sensing equipment of different tasks may also include one or more data of the same dimension. The number of the same dimensions and the corresponding dimensions to be determined can be set according to the specific task. For example, if the prediction method based on federated learning needs to predict the ocean terrain, it needs altimetry data, sea level data, etc. Therefore, the altimetry data and the sea level data can be determined as the dimensions to be determined.

[0107] Step S103-1b: obtaining local dimensions in the local data to be predicted of each remote sensing device, and determining whether the local dimensions include a plurality of identical dimensions based on the dimensions to be determined;

[0108] Parse the local data to be predicted to obtain the local dimensions in the local data to be predicted of each remote sensing device. Then compare the local dimensions with the dimensions to be determined. Specifically, first determine whether the number of the local dimensions is greater than or equal to the number of the dimensions to be determined; if the number of the local dimensions is greater than or equal to the number of the dimensions to be determined, then further compare the contents of the local dimensions with the specific dimensions of the dimensions to be determined. If the local dimensions include the contents of the specific dimensions of the dimensions to be determined, then determine that the local dimensions include the several dimensions to be determined.

[0109] Step S103-1c: if the local dimensions all include the plurality of dimensions to be determined, determining that the local dimensions of each remote sensing device include a plurality of identical dimensions;

[0110] On the contrary, if the local dimension does not completely include the several dimensions to be determined, it is determined that the local dimension of each of the remote sensing devices does not include several identical dimensions.

[0111] Step S103-1d: if the local dimensions of each of the remote sensing devices include a plurality of identical dimensions, searching the local data to be predicted for target data corresponding to the plurality of dimensions to be determined;

[0112] On the contrary, if the local dimensions of each of the remote sensing devices do not include several identical dimensions, the corresponding local data to be predicted is determined to be invalid data, and the invalid data is ignored. It can be understood that if the local data of each of the remote sensing devices do not include several target data of the same dimension, the prediction result may be biased due to incomplete information of the local data.

[0113] Step S103 - 1e: If target data corresponding to the several dimensions to be determined exist in the local data of each remote sensing device, it is determined that the local data to be predicted of each remote sensing device includes several target data of the same dimensions.

[0114] If the local data to be predicted of each of the remote sensing devices includes a plurality of target data of the same dimension, the following steps are executed: predicting the local data to be predicted based on the federated sub-model to obtain a prediction label.

[0115] This embodiment uses the above scheme to train the initial model according to the local training data to obtain a unilateral prediction model; transmit the unilateral prediction model to the ground information center, and the ground information center aggregates several unilateral prediction models corresponding to several remote sensing devices to obtain a federated prediction model, which includes several federated sub-models; receive the corresponding federated sub-models, and predict the local data to be predicted based on the federated sub-models to obtain prediction labels. Therefore, by training the model locally on the remote sensing device and realizing the comprehensive utilization of high-altitude data based on federated learning, the technical bottleneck of data transmission is broken.

[0116] Based on the above Figure 2 Based on the second embodiment shown, a third embodiment of the present invention is proposed. Figure 4 As shown, Figure 4 It is a flow chart of the second embodiment of the prediction method based on federated learning of the present invention.

[0117] The step of training the initial model according to the local training data to obtain the unilateral prediction model includes:

[0118] Step S101a: randomly obtaining initial model parameters, using the initial model parameters to obtain initial predicted labels of the local training data, and calculating a loss function based on the initial predicted labels and actual labels of the local training data;

[0119] The initial model can be a convolutional neural network model, a deep learning model, a decision tree, etc. The model parameters of each layer of the initial model are randomly obtained and saved as the initial model parameters. The initial model parameters can also be determined based on experience.

[0120] In this embodiment, a large amount of local data is obtained, and the actual label of the local data is marked, and the actual label is determined according to the result of the local data. For example, for an oil survey task, if the result of the local data is that oil exists in the area corresponding to the image, the actual label can be 1; if the result of the local data is that oil does not exist in the area corresponding to the image, the actual label can be 0; if it is impossible to determine whether the result of the local data is that oil exists in the area corresponding to the image, the actual label can be other.

[0121] After preprocessing the local data, corresponding local training data is obtained. The local data is image data, and the preprocessing includes operations such as noise reduction, geometric correction, image enhancement, and image cropping.

[0122] The local training data is processed based on the initial model parameters to obtain the initial predicted label. A loss function is calculated based on the initial predicted label and the actual label. The loss function may be a commonly used loss function for the task. In this embodiment, the cross entropy loss function is calculated based on mini-batch.

[0123] Step S101b: Based on the loss function, update the model parameters by gradient descent;

[0124] The gradients corresponding to the parameters in the initial model are calculated according to the cross entropy loss function, and the parameters are updated accordingly according to the gradients of the parameters, that is, the parameters of the initial model are adjusted. Here, the process of updating the model parameters according to the cross entropy loss function is similar to the existing model parameter updating process, and will not be described in detail here.

[0125] Step S101c: If the convergence condition is reached, the update is stopped and the corresponding model parameters are saved as the final model parameters, so as to obtain a unilateral prediction model.

[0126] Determine whether the cross entropy loss function converges, and if the cross entropy loss function converges, determine that the corresponding quality inspection model converges. The convergence condition is that the cross entropy loss function obtains a minimum value.

[0127] If the initial model is in a convergence state, the training is stopped, the parameters of the last training are saved as the final model parameters, and the unilateral prediction model is obtained based on the final model parameters.

[0128] On the contrary, if the unilateral prediction model has not reached a convergence state, the training continues: the iterative update is continuously performed until convergence, and the unilateral prediction model is finally obtained.

[0129] Based on the above scheme, this embodiment randomly obtains initial model parameters, uses the initial model parameters to obtain the initial predicted labels of the local training data, calculates the loss function based on the initial predicted labels and the actual labels of the local training data; based on the loss function, updates the model parameters by gradient descent; stops updating if the convergence condition is reached, and saves the corresponding model parameters as the final model parameters, thereby obtaining a unilateral prediction model. Therefore, model training is performed through local training data, realizing the comprehensive utilization of high-altitude data and breaking the technical bottleneck of data transmission.

[0130] In addition, this embodiment also provides a prediction device based on federated learning. Figure 5 , Figure 5 This is a schematic diagram of the functional modules of the first embodiment of the prediction device based on federated learning of the present invention.

[0131] In this embodiment, the prediction device based on federated learning is a virtual device stored in Figure 1 The memory 1005 of the remote sensing device shown is used to implement all functions of the prediction program based on federated learning: used to train the initial model according to local training data to obtain a unilateral prediction model; used to transmit the unilateral prediction model to the ground information center, and the ground information center aggregates several unilateral prediction models corresponding to several remote sensing devices to obtain a federated prediction model, and the federated prediction model includes several federated sub-models; used to receive corresponding federated sub-models, and predict the local data to be predicted based on the federated sub-models to obtain prediction labels.

[0132] Specifically, the prediction device based on federated learning includes:

[0133] A training module 10 is used to train the initial model according to local training data to obtain a unilateral prediction model;

[0134] A transmission module 20, used for transmitting the unilateral prediction model to a ground information center, and the ground information center aggregates a plurality of the unilateral prediction models corresponding to a plurality of remote sensing devices to obtain a federated prediction model, wherein the federated prediction model includes a plurality of federated sub-models;

[0135] The prediction module 30 is used to receive the corresponding federated sub-model, and predict the local data to be predicted based on the federated sub-model to obtain a prediction label.

[0136] Furthermore, the training module includes:

[0137] A computing unit, configured to randomly obtain initial model parameters, obtain initial predicted labels of the local training data using the initial model parameters, and calculate a loss function based on the initial predicted labels and actual labels of the local training data;

[0138] An updating unit, used for updating the model parameters by gradient descent based on the loss function;

[0139] The saving unit is used to stop updating if the convergence condition is reached, and save the corresponding model parameters as the final model parameters, so as to obtain a unilateral prediction model.

[0140] Furthermore, the prediction module also includes:

[0141] A judging unit, used for judging whether the local data to be predicted of each of the remote sensing devices includes a plurality of target data of the same dimension;

[0142] The execution unit is used for, if the local data to be predicted of each remote sensing device includes a plurality of target data of the same dimension, executing the steps of: predicting the local data to be predicted based on the federated sub-model to obtain a prediction label.

[0143] Furthermore, the training module also includes

[0144] A receiving unit, configured to receive instructions sent by the ground information center through a control system, execute tasks according to the instructions, and obtain the local data, wherein the local data includes image data;

[0145] The preprocessing unit is used to preprocess the local data to obtain local training data.

[0146] Furthermore, the judging unit includes:

[0147] A determination subunit, used to determine a number of dimensions to be determined based on the prediction task;

[0148] An acquisition subunit, configured to acquire local dimensions in the local data to be predicted of each remote sensing device, and determine whether the local dimensions include a plurality of identical dimensions based on the dimensions to be determined;

[0149] A first determination subunit, configured to determine that the local dimensions of each of the remote sensing devices include a plurality of identical dimensions if the local dimensions all include the plurality of dimensions to be determined;

[0150] A searching unit, configured to search for target data corresponding to the plurality of dimensions to be determined from the local data to be predicted if the local dimensions of each of the remote sensing devices include a plurality of identical dimensions;

[0151] The second determination subunit is used to determine that the local data to be predicted of each remote sensing device includes target data of the same dimension if target data corresponding to the dimensions to be determined exist in the local data of each remote sensing device.

[0152] Furthermore, the preprocessing unit comprises:

[0153] Preprocessing subunit: preprocess the local data according to the process, and save the preprocessed local data as the local training data.

[0154] Furthermore, the prediction module includes:

[0155] A receiving unit, used for receiving the federation sub-model returned by the ground information center;

[0156] The input unit is used to input the local data to be predicted into the federated sub-model, and the federated sub-model outputs the corresponding prediction label.

[0157] In addition, an embodiment of the present invention further provides a computer storage medium, on which a prediction program based on federated learning is stored. When the prediction program based on federated learning is executed by a processor, the steps of the prediction method based on federated learning as described above are implemented, which will not be repeated here.

[0158] Compared with the prior art, the present invention proposes a prediction method, device, storage medium and remote sensing equipment based on federated learning, which includes: receiving the federated sub-model sent by the ground information center; inputting local data into the federated sub-model, and the federated sub-model outputs the corresponding prediction label. Therefore, based on federated learning, the local data of the remote sensing equipment system is used for prediction, breaking the technical bottleneck of data transmission and realizing the comprehensive utilization of high-altitude data.

[0159] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0160] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0161] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present invention.

[0162] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A prediction method based on federated learning, characterized in that: The method comprises: receiving instructions sent by a ground information center through a control system, executing tasks according to the instructions, and obtaining local data, wherein the local data includes image data; Preprocessing the local data to obtain local training data, wherein the preprocessing includes noise reduction, geometric correction, image enhancement and image cropping; Randomly obtain initial model parameters, use the initial model parameters to obtain initial predicted labels of the local training data, and calculate a loss function based on the initial predicted labels and actual labels of the local training data; Based on the loss function, updating the model parameters by gradient descent; If the convergence condition is reached, the update is stopped and the corresponding model parameters are saved as the final model parameters, thereby obtaining a unilateral prediction model; The unilateral prediction model is transmitted to a ground information center, and the ground information center aggregates a plurality of the unilateral prediction models corresponding to a plurality of remote sensing devices to obtain a federated prediction model, wherein the federated prediction model includes a plurality of federated sub-models; Receiving the corresponding federated sub-model, and predicting the local data to be predicted based on the federated sub-model to obtain a prediction label; The plurality of remote sensing devices have different inclination angles, and the step of predicting the local data to be predicted based on the federated sub-model and obtaining the prediction label also includes: Determining whether the local data to be predicted of each of the remote sensing devices includes a plurality of target data of the same dimension; If the local data to be predicted of each of the remote sensing devices includes a plurality of target data of the same dimension, the steps are performed: predicting the local data to be predicted based on the federated sub-model to obtain a prediction label; The step of determining whether the local data to be predicted of each remote sensing device includes a plurality of target data of the same dimension comprises: Determine several dimensions to be judged based on the prediction task; Acquire local dimensions in the local data to be predicted of each remote sensing device, and determine whether the local dimensions include a plurality of identical dimensions based on the dimensions to be determined; If the local dimensions all include the several dimensions to be determined, then determining that the local dimensions of each remote sensing device include several identical dimensions; If the local dimensions of each of the remote sensing devices include a plurality of identical dimensions, searching the local data to be predicted for target data corresponding to the plurality of dimensions to be determined; If target data corresponding to the plurality of dimensions to be determined exist in the local data of each remote sensing device, it is determined that the local data to be predicted of each remote sensing device includes target data of the same dimensions.

2. The method according to claim 1, characterized in that The step of preprocessing the local data to obtain local training data includes: The local data is preprocessed according to the process, and the preprocessed local data is saved as the local training data.

3. The method according to claim 1, characterized in that The step of receiving the corresponding federated sub-model, predicting the local data to be predicted based on the federated sub-model, and obtaining the prediction label includes: Receiving the federation sub-model returned by the ground information center; The local data to be predicted is input into the federated sub-model, and the federated sub-model outputs the corresponding prediction label.

4. A prediction device based on federated learning, characterized in that: The prediction device based on federated learning includes: A training module is used to randomly obtain initial model parameters, use the initial model parameters to obtain initial predicted labels of local training data, calculate a loss function based on the initial predicted labels and actual labels of the local training data; based on the loss function, update the model parameters by gradient descent; if the convergence condition is reached, stop updating, and save the corresponding model parameters as final model parameters, so as to obtain a unilateral prediction model; A transmission module, used for transmitting the unilateral prediction model to a ground information center, and the ground information center aggregates a plurality of the unilateral prediction models corresponding to a plurality of remote sensing devices to obtain a federated prediction model, wherein the federated prediction model includes a plurality of federated sub-models; A prediction module, used for receiving a corresponding federated sub-model, and predicting the local data to be predicted based on the federated sub-model to obtain a prediction label; Wherein, the prediction device based on federated learning includes: receiving instructions sent by a ground information center through a control system, executing tasks according to the instructions, and obtaining local data, wherein the local data includes image data; Preprocessing the local data to obtain local training data, wherein the preprocessing includes noise reduction, geometric correction, image enhancement and image cropping; The prediction device based on federated learning also includes: Determining whether the local data to be predicted of each of the remote sensing devices includes a plurality of target data of the same dimension; If the local data to be predicted of each of the remote sensing devices includes a plurality of target data of the same dimension, the steps are performed: predicting the local data to be predicted based on the federated sub-model to obtain a prediction label; The step of determining whether the local data to be predicted of each remote sensing device includes a plurality of target data of the same dimension comprises: Determine several dimensions to be judged based on the prediction task; Acquire local dimensions in the local data to be predicted of each remote sensing device, and determine whether the local dimensions include a plurality of identical dimensions based on the dimensions to be determined; If the local dimensions all include the several dimensions to be determined, then determining that the local dimensions of each remote sensing device include several identical dimensions; If the local dimensions of each of the remote sensing devices include a plurality of identical dimensions, searching the local data to be predicted for target data corresponding to the plurality of dimensions to be determined; If target data corresponding to the several dimensions to be determined exist in the local data of each remote sensing device, it is determined that the local data to be predicted of each remote sensing device includes several target data of the same dimensions.

5. A remote sensing device, characterized in that: The remote sensing device includes a processor, a memory, and a prediction program based on federated learning stored in the memory. When the prediction program based on federated learning is executed by the processor, the steps of the prediction method based on federated learning as described in any one of claims 1 to 3 are implemented.

6. A computer storage medium, characterized in that: The computer storage medium stores a prediction program based on federated learning, and when the prediction program based on federated learning is executed by the processor, the steps of the prediction method based on federated learning as described in any one of claims 1 to 3 are implemented.

Images