A federated incremental learning system and method for edge intelligence

Abstract

The present invention discloses a federated incremental learning framework and method for edge intelligence. The framework includes: a central cloud server, multiple edge servers and multiple edge intelligent devices, the central cloud server is connected to multiple edge servers, and one edge server is connected to multiple edge intelligent devices, wherein the central cloud server includes a first initial neural network model and a second initial neural network model. The present invention solves the technical problems that the performance and resource requirements of edge devices are too high and they cannot run efficiently on edge devices with low computing and storage resources, the catastrophic forgetting problem of the incremental learning method, when the model receives new category data, its performance on the old category will drop sharply, and there is a lack of effective protection of data privacy, especially in federated learning, due to the uneven distribution of data on different edge devices, the model is prone to leaking sensitive information of local data.

Description

Technical Field

[0001] The present invention relates to the field of edge computing technology, and in particular to a federated incremental learning system and method for edge intelligence. Background Art

[0002] With the development of deep learning, artificial intelligence (AI) has garnered widespread attention in the field of edge computing. However, the characteristics of edge devices present two major challenges: the conflict between high precision and strong data privacy requirements, and the conflict between high computing resource consumption and low performance of edge devices. While existing federated learning (FL) technologies offer solutions to address privacy protection and communication costs, most schemes assume a fixed and known data distribution. In practical applications, this assumption is overly idealistic and cannot cope with the ever-changing data streams on edge devices. Summary of the Invention

[0003] The embodiments of the present invention provide a federated incremental learning system and method for edge intelligence, so as to at least solve the problems that the performance and resource requirements of edge devices are too high and they cannot run efficiently on edge devices with low computing and storage resources. The incremental learning method has the problem of catastrophic forgetting. When the model receives new category data, its performance on the old category will drop sharply. There is a lack of effective protection for data privacy. Especially in federated learning, due to the uneven distribution of data on different edge devices, the model is prone to leaking sensitive information of local data.

[0004] According to one aspect of an embodiment of the present invention, a federated incremental learning framework learning system for edge intelligence is provided. The framework system may include: a central cloud server, multiple edge servers and multiple edge intelligent devices, the central cloud server is connected to multiple edge servers, and one edge server is connected to multiple edge intelligent devices, wherein the central cloud server includes a first initial neural network model and a second initial neural network model; the central cloud server is used to train the first initial neural network model through an initial data set to obtain a trained initial feature extractor, the weight of the initial feature extractor, the initial classifier and the weight of the initial classifier, obtain the initialized feature autocorrelation matrix corresponding to the initial data set, add an expansion layer after the initial feature extractor, replace the initial classifier with a parsing classifier, obtain a second initial neural network model, train the second initial neural network model through the initial data set to obtain a trained target feature extractor, the weight of the target feature extractor, the parsing classifier and the weight of the parsing classifier, and send the target feature extractor, the weight of the target feature extractor, the parsing classifier and the weight of the parsing classifier to multiple edge servers; the edge intelligent device is used to send a number of images of multiple target time periods collected to the corresponding edge server; the edge server is used to receive the first image sent by the first edge intelligent device. The method comprises the steps of: receiving a plurality of images of a target time period, extracting features of the images through a target feature extractor, learning the extracted features through analytic incremental learning, and then updating the analytic classifier; the process of analytic incremental learning of the extracted features is to update the initial feature autocorrelation matrix; updating the weight of the analytic classifier based on the updated initial feature autocorrelation matrix and recording the number of the plurality of images of the first target time period; and finally receiving the plurality of images of the last target time period of the last edge smart device corresponding to the edge server, obtaining the weight of the analytic classifier of the last target time period of the last edge smart device and the total number of images of each edge smart device, and uploading them to the central cloud server; the central cloud server is further used to generate the weight of the global analytic classifier by weighted averaging the weight of the analytic classifier of the last target time period of each edge server and the total number of images, and sending the weight of the global analytic classifier to the plurality of edge servers, so that when each edge server processes the plurality of images of one target time period of the plurality of target time periods obtained in the next round, the weight of the analytic classifier of the last target time period of the previous round is replaced by the weight of the global analytic classifier.

[0005] Optionally, the expression for obtaining the initialization characteristic autocorrelation matrix corresponding to the initial data set is:

[0006]

[0007] in, is the number of data participating in the p-1 stage, is the high-dimensional feature of the i-th data in the p-1 stage, is the transposition of the high-dimensional features of the i-th data in the p-1 stage, represents the regularization coefficient, is the initialization feature autocorrelation matrix, I is the unit matrix, the p-1 stage is the stage corresponding to the first initial neural network model, and (0) is used to represent the stage corresponding to the initial data set as 0.

[0008] Optionally, the expression of the weight of the parsing classifier is:

[0009]

[0010] in, is the high-dimensional feature of the initial data set, is the transpose of the high-dimensional features of the initial data set, -1 is the matrix inversion operation, is the data label of the initial dataset, is the weight of the parsing classifier corresponding to the initial dataset.

[0011] Optionally, the objective loss function of the second initial neural network model is expressed as:

[0012]

[0013] Among them, FNC is the second initial convolutional neural network, F is the F norm, is the regularization term, is the initial loss function of the second initial neural network model.

[0014] Optionally, the expression for updating the initialized characteristic autocorrelation matrix is:

[0015]

[0016] in, For The updated initialized feature autocorrelation matrix corresponding to the stage, For The updated initialized feature autocorrelation matrix corresponding to the stage, is the transpose of the high-dimensional features of the i-th data in stage p, is the high-dimensional feature of the i-th data in stage p, where the P-th stage corresponds to the stage in which the edge server receives several images of a target time period sent by the edge intelligent device.

[0017] Optionally, the expression for updating the weight of the parsing classifier is:

[0018]

[0019] in, is the weight of the parsing classifier after the P-order update, is the weight of the parsing classifier after updating in the P-1 phase, is the transposition of the high-dimensional features of several images of a target time period obtained in stage p, is the high-dimensional features of several images of a target time period obtained in stage p, is the label of several images of a target time period obtained in stage p, is the weight of the parsing classifier after updating in the P-1 phase.

[0020] According to one aspect of an embodiment of the present invention, a federated incremental learning method for edge intelligence is provided. The method may include: constructing a federated incremental learning framework learning system for edge intelligence, the framework system including a central cloud server, multiple edge servers and multiple edge intelligent devices, the central cloud server being connected to multiple edge servers, and one edge server being connected to multiple edge intelligent devices, wherein the central cloud server includes a first initial neural network model and a second initial neural network model; the central cloud server is used to train the first initial neural network model using an initial data set to obtain a trained initial feature extractor, weights of the initial feature extractor, an initial classifier and weights of the initial classifier, obtain an initialized feature autocorrelation matrix corresponding to the initial data set, add an expansion layer after the initial feature extractor, replace the initial classifier with a parsing classifier, obtain a second initial neural network model, train the second initial neural network model using the initial data set to obtain a trained target feature extractor, weights of the target feature extractor, parsing classifier and weights of the parsing classifier, and send the target feature extractor, weights of the target feature extractor, parsing classifier and weights of the parsing classifier to multiple edge servers; the edge intelligent device is used to send a plurality of images collected for multiple target time periods to the corresponding edge server; the edge server is used to receive the first The edge intelligent device sends several images of the first target time period, extracts image features through the target feature extractor, and updates the parsing classifier after learning the extracted features through parsing incremental learning. The process of parsing incremental learning the extracted features is to update the initialization feature autocorrelation matrix, and use a recursive formula based on the updated initialization feature autocorrelation matrix to update the weight of the parsing classifier and record the number of several images in a target time period until the last target time period of the last edge intelligent device corresponding to the edge server is received. The weight of the parsing classifier of the last target time period of the last edge intelligent device and the total number of images of each edge intelligent device are obtained and uploaded to the central cloud server; the central cloud server is further used to generate the weight of the global parsing classifier by weighted averaging the weight of the parsing classifier of the last target time period of each edge server and the total number of images, and send the weight of the global parsing classifier to multiple edge servers, so that when each edge server processes several images of one target time period of multiple target time periods obtained in the next round, the weight of the parsing classifier of the last target time period of the previous round is replaced by the weight of the global parsing classifier.

[0021] Beneficial effects of the present invention:

[0022] 1. Continuous update capability: The framework system supports continuous updates of the model and is suitable for edge devices with constantly changing data distribution.

[0023] 2. Low resource consumption: Through local parsing incremental learning, it reduces the computing and storage requirements of edge devices and is suitable for resource-constrained devices such as drones and wearable devices.

[0024] 3. Privacy protection mechanism: The framework system achieves data privacy protection by relatively forgetting incremental data on edge devices, saving storage space while avoiding the risk of data leakage. It is particularly suitable for scenarios with high data security requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0026] Figure 1 This is a schematic diagram of a federated incremental learning framework learning system for edge intelligence according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or inherent to these processes, methods, products or devices.

[0029] Example 1

[0030] According to an embodiment of the present invention, a federated incremental learning framework learning system for edge intelligence is provided. It should be noted that the steps shown in the schematic diagram of the accompanying drawings can be executed in a computer system containing at least one set of computer-executable instructions, and although a logical order is shown in the schematic diagram, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] Figure 1 is a schematic diagram of a federated incremental learning framework learning system for edge intelligence according to an embodiment of the present invention. Figure 1 As shown, the framework system may include: a central cloud server, multiple edge servers and multiple edge smart devices, the central cloud server is connected to the multiple edge servers, and one edge server is connected to the multiple edge smart devices, wherein the central cloud server includes a first initial neural network model and a second initial neural network model. The framework system may include the following steps:

[0032] Step S101, the central cloud server is used to train the first initial neural network model through the initial data set to obtain the trained initial feature extractor, the weight of the initial feature extractor, the initial classifier and the weight of the initial classifier, obtain the initialized feature autocorrelation matrix corresponding to the initial data set, add an extension layer after the initial feature extractor, replace the initial classifier with the parsing classifier, and obtain the second initial neural network model, train the second initial neural network model through the initial data set to obtain the trained target feature extractor, the weight of the target feature extractor, the parsing classifier and the weight of the parsing classifier, and send the target feature extractor, the weight of the target feature extractor, the parsing classifier and the weight of the parsing classifier to multiple edge servers.

[0033] In the technical solution provided in step S101 of the present invention, Figure 1As shown, the central cloud server uses an initial data set (large basic data set) to train the first initial neural network model, wherein the first initial convolutional neural network model includes an initial feature extractor, an initial classifier (basic parsing classifier), and the basic parsing classifier is trained by the parsing incremental learning algorithm. After the training, the trained initial feature extractor, the weight of the initial feature extractor, the initial classifier and the weight of the initial classifier are obtained, and the initialized feature autocorrelation matrix corresponding to the initial data set is obtained. An expansion layer is added after the initial feature extractor of the first convolutional neural network model, and the initial classifier is replaced by the parsing classifier to obtain a second initial neural network model. The second initial neural network model is trained by the initial data set to obtain the trained target feature extractor, the weight of the target feature extractor, the parsing classifier and the weight of the parsing classifier. The target feature extractor, the weight of the target feature extractor, the parsing classifier and the weight of the parsing classifier are sent to each edge server, for example, Figure 1 There is a central server and multiple edge servers.

[0034] In step S102 , the edge intelligent device is configured to send the collected images of the multiple target time periods to the corresponding edge server.

[0035] In the technical solution provided in the above step S102 of the present invention, the edge intelligent device sends the collected images of multiple target time periods to the corresponding edge server, and the 10 edge intelligent devices corresponding to edge server 1; for example, the target time period can be every half hour, the first batch of data includes 10 target time periods, one target time period includes 10 half hours, the first half hour of the first target time period includes 20 images, the second half hour includes 15 images... the tenth half hour includes 16 images; the first edge intelligent device sends the first batch of data to edge server 1, and the other edge intelligent devices corresponding to edge server 1 obtain data in the same way as the first edge intelligent device, and send the obtained data to edge server 1.

[0036] Step S103, the edge server is used to receive several images of the first target time period sent by the first edge intelligent device, extract the features of the image through the target feature extractor, and update the parsing classifier after learning the extracted features through parsing incremental learning. The process of parsing the extracted features through parsing incremental learning is to update the initialization feature autocorrelation matrix, and use the recursive formula based on the updated initialization feature autocorrelation matrix to update the weight of the parsing classifier and record the number of several images of the first target time period until the last target time period of the last edge intelligent device corresponding to the edge server is received, and the weight of the parsing classifier of the last target time period of the last edge intelligent device and the total number of images of each edge intelligent device are obtained, and uploaded to the central cloud server.

[0037] In the technical solution provided in the above step S103 of the present invention, for example, the edge server 1 receives 20 images of the first target time period sent by the first edge intelligent device corresponding to the edge server, extracts features of the 20 images through the target feature extractor, and updates the parsing classifier after learning the extracted features through parsing incremental learning. The process of parsing incremental learning the extracted features is to update the initialization feature autocorrelation matrix, and based on the updated initialization feature autocorrelation matrix, the weight of the parsing classifier is updated using a recursive formula and the number of the 20 images of the first target time period (20) is recorded until the last target time period of the last edge intelligent device (the 10th) corresponding to the edge server is received. Several images (16 images) of the last target time period of the last edge intelligent device (the 10th) are obtained, and the weight of the parsing classifier of the last target time period of the last edge intelligent device and the total number of images of each edge intelligent device (that is, the total number of images of the 10 target time periods) are obtained, and uploaded to the central cloud server.

[0038] In step S104, the central cloud server is further configured to generate a weight of a global parsing classifier by weighted averaging the weight of the parsing classifier of the last target time period of each edge server and the total number of images, and to send the weight of the global parsing classifier to multiple edge servers, so that when each edge server processes several images of one target time period of multiple target time periods obtained in the next round, the weight of the parsing classifier of the last target time period of the previous round is replaced by the weight of the global parsing classifier.

[0039] In the technical solution provided in the above step S104 of the present invention, the central cloud server generates the weight of the global parsing classifier by weighted averaging the weights of the parsing classifiers received from each edge server in the last target time period and the total number of images (the sum of the total number of images obtained by each edge server) .

[0040] The weight of the parsing classifier of each edge server in the last target time period and the total number of images are weighted averaged to generate the weight of the global parsing classifier:

[0041]

[0042] in, is the weight of the global parsing classifier, , represents the amount of data received by edge server i in stage p-1. Here, stage p-1 is the stage before stage p. Since the cycle is always in progress, stage p-1 can be the stage corresponding to the initial data set or the stage before stage p in the cycle. That is, the image currently received in a target time period can be stage p. Indicates the edge server in the p-1 phase The parsing classifier weight of , in the embodiment of the present invention, the value range of P starts from 1.

[0043] The above method of this embodiment is further introduced below.

[0044] As an optional embodiment, in step S101, the expression for obtaining the initialization characteristic autocorrelation matrix corresponding to the initial data set is:

[0045]

[0046] in, is the number of data participating in the p-1 stage, is the high-dimensional feature of the i-th data in the p-1 stage, is the transposition of the high-dimensional features of the i-th data in the p-1 stage, represents the regularization coefficient, is the initialization feature autocorrelation matrix, I is the unit matrix, the p-1 stage is the stage corresponding to the first initial neural network model, and (0) is used to represent the stage corresponding to the initial data set as 0.

[0047] In this embodiment, p is 1. When p is 1, is 0, which is the stage corresponding to the initial data set, and p is 1, which is the stage of the image of the first target time period of the first edge smart device corresponding to edge server 1 obtained by edge server 1.

[0048] As an optional embodiment, in step S101, the expression of the weight of the parsing classifier is:

[0049]

[0050] in, is the high-dimensional feature of the initial data set, is the transpose of the high-dimensional features of the initial data set, -1 is the matrix inversion operation, is the data label of the initial dataset, is the weight of the parsing classifier corresponding to the initial dataset.

[0051] In this embodiment, according to the above formula, the weights corresponding to the initial data set in the second convolutional neural network are obtained.

[0052] As an optional embodiment, in step S103, the expression of the target loss function of the second initial neural network model is:

[0053]

[0054] Among them, FNC is the second initial convolutional neural network, F is the F norm, is the regularization term, is the initial loss function of the second initial neural network model.

[0055] In this embodiment, the target loss function of the second initial convolutional neural network is determined by the regularization term and the initial loss function.

[0056] As an optional embodiment, in step S103, the expression for updating the initialized characteristic autocorrelation matrix is:

[0057]

[0058] in, For The updated initialized feature autocorrelation matrix corresponding to the stage is, For The updated initialized feature autocorrelation matrix corresponding to the stage is, is the transpose of the high-dimensional features of the i-th data in stage p, is the high-dimensional feature of the i-th data in the p-th stage, where the P-th stage corresponds to the stage where the edge server receives several images of a target time period sent by the edge intelligent device.

[0059] In this embodiment, by After the stage of updating, the initial feature autocorrelation matrix is ​​updated The stage corresponds to the initialized feature autocorrelation matrix. When the edge server 1 obtains the stage of the image of the second target time period of the first edge intelligent device corresponding to the edge server 1, the P stage is the stage of the image of the second target time period of the first edge intelligent device corresponding to the edge server 1 obtained by the edge server 1, and the p-1 stage is the stage of the image of the first target time period of the first edge intelligent device corresponding to the edge server 1 obtained by the edge server 1, and the cycle is repeated.

[0060] As an optional embodiment, in step S103, the expression for updating the weight of the parsing classifier is:

[0061]

[0062] in, is the weight of the parsing classifier after the P-order update, is the weight of the parsing classifier after updating in the P-1 phase, is the transposition of the high-dimensional features of several images of a target time period obtained in stage p, is the high-dimensional features of several images of a target time period obtained in stage p, is the label of several images of a target time period obtained in stage p, is the weight of the parsing classifier after updating in the P-1 phase.

[0063] In this embodiment, by The weights of the updated parsing classifier are updated The weight of the parsing classifier corresponding to the stage, such as Figure 1 As shown, the parsing classifiers corresponding to the first edge service device are 11 to parsing classifier 1i, and each parsing classifier corresponds to a parsing classifier weight.

[0064] In an embodiment of the present invention, a central cloud server is used to train a first initial neural network model using an initial data set to obtain a trained initial feature extractor, the weight of the initial feature extractor, the initial classifier, and the weight of the initial classifier, obtain an initialized feature autocorrelation matrix corresponding to the initial data set, add an expansion layer after the initial feature extractor, replace the initial classifier with a parsing classifier, and obtain a second initial neural network model. The second initial neural network model is trained using the initial data set to obtain a trained target feature extractor, the weight of the target feature extractor, the parsing classifier, and the weight of the parsing classifier, and the target feature extractor, the weight of the target feature extractor, the parsing classifier, and the weight of the parsing classifier are sent to multiple edge servers; an edge intelligent device is used to collect a plurality of images of multiple target time periods. The image is sent to the corresponding edge server; the edge server is used to receive several images of the first target time period sent by the first edge intelligent device, extract the features of the image through the target feature extractor, and update the parsing classifier after learning the extracted features through parsing incremental learning. The process of parsing incremental learning is to update the initialization feature autocorrelation matrix, and use the recursive formula based on the updated initialization feature autocorrelation matrix to update the weight of the parsing classifier and record the number of several images of the first target time period until the last target time period of the last edge intelligent device corresponding to the edge server is received. The weight of the parsing classifier of the last target time period of the last edge intelligent device and the total number of images of each edge intelligent device are obtained, and uploaded to the central cloud server;The central cloud server is also used to generate the weight of the global parsing classifier by weighted averaging the weight of the parsing classifier of the last target time period of each edge server and the total number of images, and send the weight of the global parsing classifier to multiple edge servers, so that each edge server replaces the weight of the parsing classifier of the last target time period of the previous round with the weight of the global parsing classifier when processing several images of a target time period of multiple target time periods obtained in the next round. This solves the problem that the performance and resource requirements of edge devices are too high and cannot be run efficiently on edge devices with low computing and storage resources. The incremental learning method has a catastrophic forgetting problem. When the model receives new category data , its performance for old categories will drop sharply, lacking effective protection for data privacy. Especially in federated learning, due to the uneven distribution of data on different edge devices, the model is prone to leaking sensitive information of local data. This achieves the technical effect of supporting continuous model updates in the framework system of the embodiment of the present invention, which is suitable for edge devices with constantly changing data distribution. Through local parsing incremental learning, the computing and storage requirements of edge devices are reduced, which is suitable for resource-constrained devices. This framework system achieves data privacy protection by relatively forgetting incremental data on edge devices, saving storage space while avoiding the risk of data leakage, and is particularly suitable for scenarios with high data security requirements.

[0065] Example 2

[0066] According to an embodiment of the present invention, a federated incremental learning method for edge intelligence is also provided. It should be noted that the federated incremental learning method for edge intelligence can be used to execute the federated incremental learning framework learning system for edge intelligence in Example 1.

[0067] A federated incremental learning method for edge intelligence includes: building a federated incremental learning framework learning system for edge intelligence, the framework system includes a central cloud server, multiple edge servers and multiple edge intelligent devices, the central cloud server is connected to multiple edge servers, and one edge server is connected to multiple edge intelligent devices, wherein the central cloud server includes a first initial neural network model and a second initial neural network model.

[0068] The central cloud server is used to train the first initial neural network model with the initial data set to obtain the trained initial feature extractor, the weight of the initial feature extractor, the initial classifier and the weight of the initial classifier, obtain the initialized feature autocorrelation matrix corresponding to the initial data set, add an extension layer after the initial feature extractor, replace the initial classifier with the parsing classifier, and obtain the second initial neural network model; train the second initial neural network model with the initial data set to obtain the trained target feature extractor, the weight of the target feature extractor, the parsing classifier and the weight of the parsing classifier; and send the target feature extractor, the weight of the target feature extractor, the parsing classifier and the weight of the parsing classifier to multiple edge servers.

[0069] The edge intelligent device is used to send several collected images of multiple target time periods to the corresponding edge server.

[0070] The edge server is used to receive several images of the first target time period sent by the first edge intelligent device, extract the features of the image through the target feature extractor, and update the parsing classifier after learning the extracted features through parsing incremental learning, wherein the process of parsing the extracted features through parsing incremental learning is to update the initialization feature autocorrelation matrix, and use the recursive formula based on the updated initialization feature autocorrelation matrix to update the weight of the parsing classifier and record the number of several images in a target time period until the last target time period of the last edge intelligent device corresponding to the edge server is received, and the weight of the parsing classifier of the last target time period of the last edge intelligent device and the total number of images of each edge intelligent device are obtained, and uploaded to the central cloud server.

[0071] The central cloud server is also used to generate the weight of the global parsing classifier by weighted averaging the weight of the parsing classifier of the last target time period of each edge server and the total number of images, and send the weight of the global parsing classifier to multiple edge servers, so that when each edge server processes several images of one target time period of multiple target time periods obtained in the next round, the weight of the parsing classifier of the last target time period of the previous round is replaced by the weight of the global parsing classifier.

[0072] In this embodiment, the central cloud server is used to train the first initial neural network model with the initial data set to obtain the trained initial feature extractor, the weight of the initial feature extractor, the initial classifier and the weight of the initial classifier, obtain the initialized feature autocorrelation matrix corresponding to the initial data set, add an expansion layer after the initial feature extractor, replace the initial classifier with the parsing classifier, and obtain the second initial neural network model; train the second initial neural network model with the initial data set to obtain the trained target feature extractor, the weight of the target feature extractor, the parsing classifier and the weight of the parsing classifier, and send the target feature extractor, the weight of the target feature extractor, the parsing classifier and the weight of the parsing classifier to multiple edge servers; the edge intelligent device is used to send a plurality of images collected for multiple target time periods to the edge servers; Send to the corresponding edge server; the edge server is used to receive several images of the first target time period sent by the first edge intelligent device, extract the features of the image through the target feature extractor, and update the parsing classifier after learning the extracted features through parsing incremental learning. The process of parsing incremental learning is to update the initialization feature autocorrelation matrix of the extracted features, and use the recursive formula based on the updated initialization feature autocorrelation matrix to update the weight of the parsing classifier and record the number of several images of the first target time period until the last target time period of the last edge intelligent device corresponding to the edge server is received. The weight of the parsing classifier of the last target time period of the last edge intelligent device and the total number of images of each edge intelligent device are obtained, and uploaded to the central cloud server;The central cloud server is also used to generate the weight of the global parsing classifier by weighted averaging the weight of the parsing classifier of the last target time period of each edge server and the total number of images, and send the weight of the global parsing classifier to multiple edge servers, so that each edge server replaces the weight of the parsing classifier of the last target time period of the previous round with the weight of the global parsing classifier when processing several images of a target time period of multiple target time periods obtained in the next round. This solves the problem that the performance and resource requirements of edge devices are too high and cannot be run efficiently on edge devices with low computing and storage resources. The incremental learning method has a catastrophic forgetting problem. When the model receives new category data , its performance for old categories will drop sharply, lacking effective protection for data privacy. Especially in federated learning, due to the uneven distribution of data on different edge devices, the model is prone to leaking sensitive information of local data. This achieves the technical effect of supporting continuous model updates in the framework system of the embodiment of the present invention, which is suitable for edge devices with constantly changing data distribution. Through local parsing incremental learning, the computing and storage requirements of edge devices are reduced, which is suitable for resource-constrained devices. This framework system achieves data privacy protection by relatively forgetting incremental data on edge devices, saving storage space while avoiding the risk of data leakage, and is particularly suitable for scenarios with high data security requirements.

[0073] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0074] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0075] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0076] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs.

[0077] In addition, the functional units in various embodiments of the present invention may be integrated into a first processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0078] The above are only preferred embodiments of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A federated incremental learning system for edge intelligence, characterized by: include: A central cloud server, multiple edge servers, and multiple edge smart devices, wherein the central cloud server is connected to the multiple edge servers, and one edge server is connected to the multiple edge smart devices, wherein the central cloud server includes a first initial neural network model and a second initial neural network model; The central cloud server is used to train the first initial neural network model using the initial data set to obtain the trained initial feature extractor, the weight of the initial feature extractor, the initial classifier, and the weight of the initial classifier, obtain the initialized feature autocorrelation matrix corresponding to the initial data set, add an expansion layer after the initial feature extractor, replace the initial classifier with the parsing classifier, and obtain the second initial neural network model, train the second initial neural network model using the initial data set to obtain the trained target feature extractor, the weight of the target feature extractor, the parsing classifier, and the weight of the parsing classifier, and send the target feature extractor, the weight of the target feature extractor, the parsing classifier, and the weight of the parsing classifier to multiple edge servers; The edge intelligent device is used to send the collected images of multiple target time periods to the corresponding edge server; The edge server is used to receive several images of the first target time period sent by the first edge smart device, extract features of the images through the target feature extractor, and update the parsing classifier after learning the extracted features through parsing incremental learning. The process of parsing the extracted features through parsing incremental learning is to update the initialization feature autocorrelation matrix, and use a recursive formula based on the updated initialization feature autocorrelation matrix to update the weight of the parsing classifier and record the number of several images of the first target time period until the last target time period of the last edge smart device corresponding to the edge server is received. The weight of the parsing classifier of the last target time period of the last edge smart device and the total number of images of each edge smart device are obtained, and uploaded to the central cloud server; The central cloud server is also used to generate the weight of the global parsing classifier by weighted averaging the weight of the parsing classifier of the last target time period of each edge server and the total number of images, and send the weight of the global parsing classifier to multiple edge servers, so that when each edge server processes several images of one target time period of multiple target time periods obtained in the next round, the weight of the parsing classifier of the last target time period of the previous round is replaced by the weight of the global parsing classifier.

2. The system according to claim 1, wherein: The expression for obtaining the initial characteristic autocorrelation matrix corresponding to the initial data set is: in, is the number of data participating in the p-1 stage, is the high-dimensional feature of the i-th data in the p-1 stage, is the transposition of the high-dimensional features of the i-th data in the p-1 stage, represents the regularization coefficient, is the initialization feature autocorrelation matrix, I is the unit matrix, the p-1 stage is the stage corresponding to the first initial neural network model, and (0) is used to represent the stage corresponding to the initial data set as 0.

3. The system according to claim 1, wherein: The expression of the weight of the parsing classifier is: in, is the high-dimensional feature of the initial data set, is the transpose of the high-dimensional features of the initial data set, -1 is the matrix inversion operation, is the data label of the initial dataset, is the weight of the parsing classifier corresponding to the initial dataset.

4. The system according to claim 3, characterized in that The expression of the objective loss function of the second initial neural network model is: Among them, FNC is the second initial convolutional neural network, F is the F norm, is the regularization term, is the initial loss function of the second initial neural network model.

5. The system according to claim 1, wherein: The expression for updating the initialization characteristic autocorrelation matrix is: in, For The updated initialized feature autocorrelation matrix corresponding to the stage, For The updated initialized feature autocorrelation matrix corresponding to the stage, is the transpose of the high-dimensional features of the i-th data in stage p, is the high-dimensional feature of the i-th data in the p-th stage, where the P-th stage corresponds to the stage where the edge server receives several images of a target time period sent by the edge intelligent device.

6. The system according to claim 5, characterized in that The expression for updating the weight of the parsing classifier is: in, is the weight of the parsing classifier after the P-order update, is the weight of the parsing classifier after updating in the P-1 phase, is the transposition of the high-dimensional features of several images of a target time period obtained in stage p, is the high-dimensional features of several images of a target time period obtained in stage p, is the label of several images of a target time period obtained in stage p, is the weight of the parsing classifier after updating in the P-1 phase.

7. A federated incremental learning method for edge intelligence, characterized in that: include: Constructing an edge intelligence federated incremental learning system, the system including a central cloud server, multiple edge servers, and multiple edge intelligent devices, wherein the central cloud server is connected to the multiple edge servers, and one edge server is connected to the multiple edge intelligent devices, wherein the central cloud server includes a first initial neural network model and a second initial neural network model; The central cloud server is used to train the first initial neural network model using the initial data set to obtain the trained initial feature extractor, the weight of the initial feature extractor, the initial classifier, and the weight of the initial classifier, obtain the initialized feature autocorrelation matrix corresponding to the initial data set, add an expansion layer after the initial feature extractor, replace the initial classifier with the parsing classifier, and obtain the second initial neural network model, train the second initial neural network model using the initial data set to obtain the trained target feature extractor, the weight of the target feature extractor, the parsing classifier, and the weight of the parsing classifier, and send the target feature extractor, the weight of the target feature extractor, the parsing classifier, and the weight of the parsing classifier to multiple edge servers; The edge intelligent device is used to send the collected images of multiple target time periods to the corresponding edge server; The edge server is used to receive several images of the first target time period sent by the first edge smart device, extract features of the images through the target feature extractor, and update the parsing classifier after learning the extracted features through analytic incremental learning, wherein the process of learning the extracted features through analytic incremental learning is to update the initialization feature autocorrelation matrix, and use a recursive formula based on the updated initialization feature autocorrelation matrix to update the weight of the parsing classifier and record the number of several images in a target time period until the last target time period of the last edge smart device corresponding to the edge server is received, and the weight of the parsing classifier of the last target time period of the last edge smart device and the total number of images of each edge smart device are obtained, and uploaded to the central cloud server; The central cloud server is also used to generate the weight of the global parsing classifier by weighted averaging the weight of the parsing classifier of the last target time period of each edge server and the total number of images, and send the weight of the global parsing classifier to multiple edge servers, so that when each edge server processes several images of one target time period of multiple target time periods obtained in the next round, the weight of the parsing classifier of the last target time period of the previous round is replaced by the weight of the global parsing classifier.

8. A computer system, characterized in that include: One or more processors, and a computer-readable storage medium for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method of claim 7.

9. A computer-readable storage medium, characterized in that Computer-executable instructions are stored, and when the instructions are executed, they are used to implement the method of claim 7.

10. A computer program product, characterized in that The invention comprises computer executable instructions, which are used to implement the method of claim 7 when the instructions are executed.

Images