A collaborative training method and device based on longitudinal federated learning and a storage medium

Abstract

The application provides a collaborative training method and device based on longitudinal federated learning and a storage medium, the split neural network comprises a sub-neural network located at each client, a plurality of client feature fusion neural networks and a feature fusion network located at a server end; a first loss function is determined by using an output result of the client feature fusion neural network and a prediction data multiplexing label Y; a first intermediate feature is output after the sub-neural network and the client feature fusion neural network are adjusted based on the first loss function, and the feature fusion network determines an output result of the feature fusion neural network based on the plurality of first intermediate features; a second loss function is determined based on the output result of the feature fusion neural network and the prediction data multiplexing label Y; and the split neural network is adjusted based on the second loss function. The network converges quickly, the client data does not have to be synchronized, the client can be dynamically expanded or deleted, the robustness is higher, and the accurate prediction of the prediction target is realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of machine learning, and particularly relates to a collaborative training method and device based on longitudinal federated learning and a storage medium. BACKGROUND

[0002] This section is intended to provide background or context to the embodiments of the application recited in the claims. The description herein does not constitute admission that the prior publication, square, or subject matter described herein and / or the material contained therein is or was prior art to the claims at issue.

[0003] In the supply chain, the enterprise manufacturer produces products in the factory and sends them to the distributor, who sends them to the downstream retailer. The manufacturer has complete shipment data, but fails to consider the actual sales and inventory of downstream distributors and retailers when predicting future demand based on this data, resulting in often inaccurate predictions. For example, predicting the number of cars produced or demanded by the enterprise next month using only internal data: sales data, inventory data, historical production data, and parts data. However, it is found that the prediction accuracy is not high enough when using only internal data. Combining data from upstream and downstream of the supply chain, such as the data from distributors, second-tier distributors, and third-tier distributors to predict production, the data from each enterprise in the supply chain can be used as an influencing feature, and more data sources can be considered for prediction to improve prediction accuracy.

[0004] In the first aspect, due to confidentiality obligations, information security, business competition, and other reasons, enterprises in the supply chain are often unwilling to share their own data, resulting in low data sharing among upstream, midstream, and downstream enterprises, and a large amount of data cannot be fully utilized, resulting in a data island problem. Therefore, how to form a collaborative training analysis method for enterprise data is an urgent problem for enterprises in the value chain.

[0005] In the second aspect, due to the different computing capabilities of different clients, when a faster client transmits data to the server, the server needs to wait for the remaining clients to transmit data to the server. After the server waits for data synchronization, the server can continue to perform calculations, and asynchronous calculations cannot be achieved. If the number of clients is large, or some clients are very slow, it will seriously affect the entire network training and prediction time.

[0006] In the third aspect, in practice, due to various reasons such as cooperation period, business interest changes, and client crashes, clients often need to dynamically exit or enter the network, but the existing network does not allow clients to dynamically exit or join. Once one or more clients exit, the server network cannot synchronize the relevant data, resulting in data synchronization problems and making the network unusable. If a client wants to join, but the server and client network in the prior art cannot be expanded, and cannot be compatible with the newly added client.

[0007] Fourthly, the prior art often requires input data to be aligned, which will result in additional calculation, and if the data is not aligned or alignment error occurs, the calculation cannot be performed

[0008] Fifthly, the network of the prior art is fixed once it is built, and for some occasions where the network size is not suitable, a large amount of calculation time is required, and the network size cannot be dynamically adjusted according to the actual situation.

[0009] Sixthly, in the prior art, the network is a whole, and when the front part of the network is calculating, the back end of the network can only wait, and different parts of a deep network cannot be asynchronously run.

[0010] Seventhly, in the prior art, the split neural network is calculated using labels at the end of the network, and then back propagation is performed, the partial derivative of the loss function with respect to the neuron parameters is calculated to obtain error information, the error information is transmitted from the server to all clients, and each client needs to perform internal back propagation after receiving the error information, and the next forward propagation cannot be performed until all clients complete the back propagation, which results in a long message synchronization time and more node transmission, consumes a large amount of time, and causes the split neural network to converge slowly. SUMMARY

[0011] In view of the problems in the prior art, a dynamic construction method of a split learning neural network and a supply chain demand prediction method and device based on longitudinal federated learning collaborative training and a storage medium are provided, which can partially or completely solve the above problems.

[0012] The present application provides the following solutions.

[0013] In an optional embodiment, the present application provides a collaborative training method based on longitudinal federated learning, and the split neural network established includes a sub-neural network located at each client and a plurality of client feature fusion neural networks and a feature fusion network located at the server end.

[0014] The server end receives the output features of each sub-neural network and inputs each output feature into the corresponding client feature fusion neural network to determine a first loss function for each client feature fusion neural network through the output result of the client feature fusion neural network and the prediction data multiplex label Y; and adjusts at least one of the sub-neural network and the client feature fusion neural network based on the first loss function.

[0015] The server end receives the first intermediate features output by each client end feature fusion neural network through the feature fusion network, and performs feature fusion based on the first intermediate features output by the plurality of client end feature fusion neural networks respectively to obtain second intermediate features; determines an output result of the feature fusion network based on the second intermediate features; and

[0016] determines a second loss function for the feature fusion network based on the output result of the feature fusion network and the prediction data multiplexing label Y; and adjusts at least one of the sub neural network, the client end feature fusion neural network and the feature fusion network based on the second loss function.

[0017] In an optional embodiment, the present application provides a collaborative training method based on longitudinal federated learning, wherein the feature fusion network comprises a plurality of group feature fusion sub-networks and a global fusion sub-network; in the case of grouping a plurality of clients, each group of clients corresponds to a group feature fusion sub-network;

[0018] For each group feature fusion sub-network in the plurality of group feature fusion sub-networks, the first intermediate features output by the client end feature fusion neural network corresponding to the client within the group are received, and the received first intermediate features are fused to obtain the second intermediate features corresponding to each group feature fusion sub-network;

[0019] The global fusion sub-network receives the second intermediate features corresponding to the plurality of group feature fusion sub-networks respectively, and determines an output result of the global fusion sub-network.

[0020] In an optional embodiment, the present application provides a collaborative training method based on longitudinal federated learning, preferably, the adjusting at least one of the sub neural network and the client end feature fusion neural network based on the first loss function further comprises:

[0021] The at least one of the sub neural network and the client end feature fusion neural network is adjusted through the back propagation of the first loss function in a loop iteration until the iteration is stopped after the sub neural network and the client end feature fusion neural network converge or the iteration threshold is reached, the client end feature fusion neural network outputs the first intermediate features and sends them to the corresponding group feature fusion sub-network.

[0022] In an optional embodiment, the present application provides a collaborative training method based on longitudinal federated learning, preferably, the adjusting at least one of the sub neural network, the client end feature fusion neural network and the feature fusion network based on the second loss function further comprises:

[0023] The at least one of the sub neural network, the client feature fusion neural network, the group feature fusion subnetwork and the global fusion subnetwork is adjusted through the back propagation of the second loss function in a loop iteration until the split neural network converges or the iteration reaches an iteration threshold, and the collaborative training of the vertical federated learning of the split neural network is completed; preferably, the at least one of the group feature fusion subnetwork and the global fusion subnetwork is adjusted through the back propagation of the second loss function in a loop iteration until the split neural network converges or the iteration reaches an iteration threshold, and the collaborative training of the vertical federated learning of the split neural network is completed.

[0024] In an optional embodiment, the application provides a collaborative training method based on vertical federated learning, further comprising: inputting original data through the sub neural network of the client, and outputting a final prediction value through the global fusion subnetwork of the server. If a client sends a request to join or exit during the prediction process, for the exiting client, the server deletes the neurons corresponding to the exiting client in the input layer of the group feature fusion neural network, and notifies the exiting client of the information about the deletion of the related neurons. For the joining client, the server re-adds neurons in the input layer of the group feature fusion neural network for the client, the added neurons copy the parameters of the neighbor neurons in the input layer of the group feature fusion neural network, the neurons are fully connected with the neurons of the next layer network, the server dynamically generates the client feature fusion neural network and the sub neural network for the corresponding client again, and sends the sub neural network to the added client. The client feature fusion neural network receives the features sent by the added sub neural network, the added client feature fusion neural network receives the features sent by the added sub neural network and generates first intermediate features after fusion processing, and sends the first intermediate features to the corresponding added neurons. Only the sub neural network and the corresponding client feature fusion neural network of the added client need to be retrained using the prediction data multiplexing label Y to obtain a new split network.

[0025] In an optional embodiment, the application provides a collaborative training method based on vertical federated learning, and the server side pre-designs a calculation time threshold. If the time spent in the iteration adjustment of the sub neural network and the client feature fusion neural network corresponding to the client exceeds the calculation time threshold, the server side deletes the neurons in the input layer of the group feature fusion neural network corresponding to the client, and sends the related indication information about the deletion of the neurons to the client corresponding to the neurons.

[0026] If the deleted client re-sends the request or a new client sends a request, the request carries the metadata and / or state marker information of the client, the server re-adds the neurons in the input layer of the group feature fusion neural network for the client, and when the server dynamically generates the client feature fusion neural network and the sub neural network for the corresponding client, at least one of the client feature fusion neural network and the sub neural network is compressed, and the existing sub neural network of the corresponding client is updated with the compressed sub neural network, the client feature fusion neural network receives the features sent by the newly added sub network, the newly added client feature fusion neural network receives the features sent by the newly added sub network and generates first intermediate features after fusion processing, and the first intermediate features are sent to the corresponding newly added neurons, and the newly added neurons can join the training process.

[0027] Alternatively, if the deleted client re-sends the request or a new client sends a request, the request carries the metadata and / or state marker information of the client, the server dynamically generates a compressed client feature fusion neural network and / or a client generates a compressed sub neural network according to its own metadata for the corresponding client, the client feature fusion neural network receives the features sent by the newly added sub network, the newly added client feature fusion neural network receives the features sent by the newly added sub network and generates first intermediate features after fusion processing, and the first intermediate features are sent to the corresponding newly added neurons, and the newly added neurons can join the training process.

[0028] In an optional embodiment, the present application provides a collaborative training method based on longitudinal federated learning. The sub neural network is generated according to the metadata of the client, so that the input layer of the sub neural network can input various raw data of the client, and the output layer of all the sub neural networks has a preset number of neurons, which is preset by the server side in advance or negotiated and set by multiple clients. Preferably, the raw data of the client is recorded in the form of a tensor or a table, including features and samples. The client can collect raw data at a time period of days, weeks, months, or half a year, or can collect or record raw data in real time. Those skilled in the art can set other collection periods or time intervals as needed. Preferably, the input data of the client's sub neural network can be raw data or intermediate data obtained after processing raw data. Preferably, the metadata includes the features, size, dimension, attributes, generation time, type, shape, client flag, variable name, and dimension of the raw data. Metadata is data that describes other data, or in other words, structured data that provides information about a certain resource. Preferably, the metadata sent by the client to the client is data describing the raw data, or data describing the intermediate data obtained after processing the raw data.

[0029] In an optional embodiment, the present application provides a collaborative training method based on longitudinal federated learning, and part or all of the clients can also run in the server.

[0030] In an optional embodiment, the present application also provides a collaborative training device based on longitudinal federated learning, comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of the embodiments of the present application.

[0031] In an optional embodiment, the present application also provides a storage medium, which is a computer readable storage medium, and the computer readable storage medium stores a program, and when the program is executed by a multi-core processor, the multi-core processor performs the method according to any one of the embodiments of the present application.

[0032] To at least partially solve one or more of the above problems and other potential problems, example embodiments of the present disclosure propose a collaborative training scheme based on longitudinal federated learning. In this scheme, 1, the input layer of the client's sub neural network is adaptively generated according to the metadata to be input by the client, which can realize the input of any type and quantity of metadata; 2, the client's sub neural network is adaptively generated according to the metadata to be input by the client, so that the time consumption of the client's sub neural network is dynamically adjustable; 3, by controlling and generating the output layer of the sub neural network in advance, the data alignment is realized through the neural network, and there is no need to specially align the original data or input data; 4, through the group neural network, the dynamic joining and exiting of the client is realized, so that the split neural network is more flexible and more suitable for actual situation; 5, through the client neural network and the global fusion neural network, the prediction data and the label Y are reused, and the split neural network is realized to quickly converge.

[0033] Other advantages of the present application will be described in more detail in conjunction with the following description and drawings.

[0034] It should be understood that the above description is only a summary of the technical solutions of the present application, so as to enable a clearer understanding of the technical means of the present application, so that the content of the description can be implemented. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are illustrated. BRIEF DESCRIPTION OF DRAWINGS

[0035] The advantages and benefits described herein, as well as additional advantages and benefits, will be appreciated by ordinary skilled persons in the art upon reading the detailed description of exemplary embodiments that follows, together with the drawings. The drawings are for purposes of illustrating exemplary embodiments and are not to be construed as limiting the application. Moreover, in the drawings, like reference numerals designate like parts throughout the several views. In the drawings:

[0036] Figure 1 A hardware architecture diagram according to an embodiment of the application;

[0037] Figure 2 A collaborative training method and device flowchart based on vertical federated learning according to an embodiment of the application;

[0038] Figure 3 A split network model architecture based on vertical federated learning according to another embodiment of the application;

[0039] Figure 4 A supply chain automobile demand prediction method flowchart based on vertical federated learning according to another embodiment of the application;

[0040] Figure 5 A split network method for collaborative training vertical federated learning to predict automobile demand according to another embodiment of the application;

[0041] Figure 6 A whole vehicle carbon emission prediction method flowchart based on collaborative training vertical federated learning according to another embodiment of the application;

[0042] Figure 7 A split network method for collaborative training vertical federated learning to predict carbon emissions according to another embodiment of the application.

[0043] In the drawings, the same or corresponding reference numerals designate the same or corresponding parts throughout the several views. DETAILED DESCRIPTION

[0044] Exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0045] In the description of embodiments of the present application, it should be understood that terms such as "include" or "have" are intended to indicate that there is existence of the features, numbers, steps, actions, components, parts or combinations thereof disclosed in the specification, and do not exclude the existence or possibility of one or more other features, numbers, steps, actions, components, parts or combinations thereof.

[0046] Unless otherwise stated, " / " means or, for example, A / B can mean A or B; "and / or" in this document merely describes an associated relationship, which means that there can be three relationships, for example, A and / or B, which means that A exists alone, A and B exist together, and B exists alone.

[0047] The terms "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second", etc. can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise stated, the meaning of "multiple" is two or more.

[0048] All codes in this application are exemplary, and those skilled in the art can think of various modifications according to the programming language used, specific needs and personal habits, etc. without departing from the idea of the present application.

[0049] In order to clearly describe the embodiments of the present application, some concepts that may appear in subsequent embodiments will be introduced first.

[0050] At present, the research on horizontal federated learning is relatively mature, but the research on vertical federated learning is still less, especially the research on vertical federated learning combined with split learning is less. Split learning (Split Neural Networks, SplitNN) is a distributed and private deep learning technology that can train deep neural networks on multiple data sources without directly sharing raw labeled data. In the existing split network, after the input data, the input data of the client enters the sub-network of the client, the split neural network is forward propagated for calculation, the sub-network outputs intermediate values and sends them to the server side, the server side synchronizes the intermediate value features of all client sides, after synchronization, the server side network is input for calculation, and finally the loss is calculated according to the label. Loss is back-propagated in the split neural network, and is iterated constantly to realize the convergence of the entire split neural network.

[0051] However, the traditional scheme has the following problems:

[0052] In the prior art, the network is a whole, and the front end of the network can only wait while the back end of the network is calculating. Different parts of a deep network cannot be run asynchronously.

[0053] Due to the reasons of confidentiality obligation, information security, business competition and the like, the upstream and downstream enterprises in the supply chain are often unwilling to share their own data, resulting in low data sharing degree of the upstream, middle and downstream enterprises, and a large amount of data cannot be fully utilized, and there is a data island problem. Therefore, how to form a data collaborative training analysis method of each enterprise is an urgent problem for each enterprise in the value chain.

[0054] Due to different computing capabilities of different clients, when a faster client transmits data to the server, the server needs to wait for the remaining clients to transmit data to the server, and after the server waits for data synchronization, the server side can continue to perform calculation, and asynchronous calculation cannot be realized. Once the number of clients is large, or part of the clients is very slow, it will seriously affect the entire network training and prediction time.

[0055] In the prior art, the network is a whole, and when the front end of the network is calculating, the rear end of the network can only wait, and different parts of a deeper network cannot be run asynchronously.

[0056] Each client's sub-network is completely the same, and often requires that the input data is aligned, without differentiating adjustment according to the input data and operation time of the client, resulting in that the input data of different clients needs to be aligned, and the data alignment will cause additional calculation amount and time consumption. If the data is not aligned or alignment error occurs, calculation cannot be performed, and the entire network is not flexible and cannot adapt to various real situations.

[0057] Once the network of the prior art is built, the network size is fixed, and for some occasions where the network size is not suitable, a large amount of calculation time is required, and the network size cannot be dynamically adjusted according to the actual situation, and the operation time of each client cannot be controlled according to actual needs.

[0058] In actual practice, due to various reasons such as cooperation period of upstream and downstream, change of business interests, client crash problem and the like, the client will dynamically exit or enter, but the existing network does not allow the client to dynamically exit or join. On the one hand, once one or more clients exit, the server side network cannot synchronize to the related data, resulting in synchronization problem. On the other hand, if a client wants to join, but the network of the server and the client in the prior art cannot be expanded, and the newly joined client cannot be compatible.

[0059] The existing split neural network calculates a loss function using labels at the end of the network, then performs back propagation, calculates the partial derivative of the loss function with respect to the neuron parameters to obtain error information, and transmits the error information from the server end to all client ends. Each client end receives the error information and further performs internal back propagation. All client ends need to complete back propagation before the next forward propagation can be performed, which results in a long message synchronization time and a large number of node transmissions, consumes a large amount of time, and causes the split neural network to converge slowly.

[0060] To at least partially address one or more of the above-referenced issues and other potential issues, example embodiments of the present disclosure propose a scheme for collaborative training based on longitudinal federated learning. In this scheme, 1, the server is informed by metadata, and the server adaptively generates a sub neural network for the client according to the situation of the original data to be input by the client. The input layer can realize input of original data of any name, type, latitude or quantity; 2, different sizes of sub neural networks are adaptively generated for different clients according to the situation of the clients, so that the time consumption of the client sub neural network is dynamically controllable; 3, data alignment is realized by controlling the output layer of the sub neural network in advance, without the need to align the data before inputting the original data; 4, dynamic joining and exiting of the client is realized through the group neural network, making the split neural network more flexible, the system more robust, the use of the network more flexible and convenient, and more in line with the actual situation; 5, different parts of the network are fully utilized to realize asynchronous operation of the network and speed up the convergence of the network; 6, the prediction data and the label Y are reused by the client neural network and the global fusion neural network, realizing fast convergence of the split neural network.

[0061] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0062] Firstly, refer to Figure 1 which schematically shows a schematic diagram of a system architecture diagram in which an example implementation according to the present disclosure can be used.

[0063] Figure 1 which shows a schematic diagram of an example of a computing device according to an embodiment of the present disclosure. It should be noted that Figure 1 which is a structural schematic diagram of a hardware running environment of a method for collaborative training based on longitudinal federated learning according to an embodiment of the present application. The client or server end device according to an embodiment of the present application can be a mobile phone, a tablet computer, a PC, an embedded device, a server, a portable computer, etc.

[0064] As Figure 1As shown, the client can include a communication module, a sub neural network training and / or generation module, a data processing module, a database, a memory, a processor such as a CPU, a network interface, a user interface, and the like, the communication module communicates with the communication module of the server through a communication bus or a wireless transmission module, the sub neural network training and / or generation module is used to generate the sub neural network of the client and / or run the sub neural network. The server can include a communication module, a split neural network training and / or generation module, a parameter management module, a database, a memory, a processor such as a CPU, a network interface, a user interface, the communication module communicates with the communication module of the client through a communication bus or a wireless transmission module, the split neural network training and / or generation module is used to generate the split neural network and run the client feature fusion neural network and the feature fusion network, wherein the feature fusion network includes a plurality of group feature fusion sub-networks and a global fusion sub-network. In an embodiment, the server generates the sub neural network, the client feature fusion neural network and the feature fusion network through the split neural network training and / or generation module, and the server sends the sub neural network to the client, the client runs the sub neural network through the sub neural network training and / or generation module, and the server runs the client feature fusion neural network and the feature fusion network through the split neural network training and / or generation module. In another embodiment, the server generates and runs the client feature fusion neural network and the feature fusion network through the split neural network training and / or generation module, and the client generates and runs the sub neural network through the sub neural network training and / or generation module. The communication bus or the wireless transmission module is used to realize the connection communication between the components. The user interface can include a human-computer interaction module, a display, an input unit such as a keyboard, and the optional user interface can further include a standard wired interface, a wireless interface, or a data input / output interface. The network interface can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory can be a high-speed RAM memory, or a stable memory (non-volatile memory) such as a disk memory. The memory can optionally be a storage device independent of the aforementioned processor.

[0065] Those skilled in the art can understand that Figure 1 The client or server device structure shown in the foregoing embodiments does not constitute a limitation on the client or server device, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0066] Embodiment 1

[0067] In the value chain, a large amount of data training analysis can save production and operation costs for enterprises, but at present, each enterprise in the value chain pays more and more attention to data privacy and security, and is unwilling to share original data with upstream and downstream enterprises, so a large amount of data cannot be fully utilized, and there is a data island problem. Metadata is data that describes information resources or data objects, and its purpose is to identify resources, evaluate resources, track changes in resources during use, manage a large amount of networked data simply and efficiently, and effectively discover, find, integrate, and manage the use of resources. Therefore, metadata is data that can be publicly disclosed. Metadata includes the characteristics, size, dimension, attributes, generation time, type, shape, client identifier, variable name and dimension of the input data. Metadata is data that describes other data, or structured data that provides information about a certain resource, which can be generated by the client or server side according to the input data or original data of each client. The original data is actual data collected at a specific period, including characteristics and samples; the original data can be collected by rules and methods specified by those skilled in the art, and the original data records the original data of the client in the form of a tensor or a table, including characteristics and samples, for example, characteristic 1 is automobile sales, and samples include the automobile sales data of different clients in January, February, March, …; characteristic 2 is automobile inventory, and samples include the automobile inventory data of different clients in January, February, March, …; characteristic 3 is accessory sales, and samples include the accessory sales data of different clients in January, February, March, …; characteristic 4 is accessory inventory, …, and so on. Those skilled in the art should understand that the characteristics and samples referred to in the present application are not limited to the above, the characteristics can be the characteristics of part or all of the data collected or recorded by each client in practice, and the samples can also be part or all of the data collected or recorded by each client in practice. The characteristics, dimensions, types, etc. of the original data of different clients that can input data can be different, and the characteristics, dimensions, types, etc. of the original data of different groups can also be different, and the client can collect the original data in units of months, weeks, or in real time, and those skilled in the art can arbitrarily set according to actual needs.

[0068] Based on the collaborative training method based on longitudinal federated learning proposed in the present application, the client only needs to share metadata, without sharing original data, which protects the data privacy and security of the enterprise, and makes it possible for enterprises in the value chain to safely carry out certain cooperation without sharing data.

[0069] This invention provides a collaborative training method based on vertical federated learning. In an optional embodiment of this invention, the method establishes a split neural network model including: a sub-neural network located on each client and multiple client feature fusion neural networks and feature fusion networks located on the server; preferably, it also includes a prediction data reuse label Y.

[0070] like Figure 2 As shown, a collaborative training device or system based on vertical federated learning consists of a server S and N clients {C1, C2, ..., C...} i ,…,C N The system consists of a server (S) and a client (C). Here, i ∈ [1, N] and i is a positive integer, with each value of i corresponding to a specific client. N is the number of clients. The order of steps indicates the sequence of events. Arrowed lines represent the nodes where steps are executed or the direction of data flow. The same step number indicates actions performed sequentially on different objects within the same step to complete the same task. i The interaction process includes the following:

[0071] Step S1: The server S sends a message to each client C. i Send a metadata retrieval request to obtain metadata information related to the task. Preferably, the task is for an automobile manufacturer to predict the number of cars it should produce next month, or to accurately predict the corresponding carbon emissions by combining component data from component manufacturers of various subsystems within the vehicle, etc.

[0072] Step S2: Each client C i Send metadata to the server. In another alternative embodiment, step S2 can also be performed by each client C. iThe server end receives the request sent by the client, and the request carries the metadata of each client, state marker information, and the state marker information is used to indicate that the client decides to apply for joining or exiting the task. Alternatively, in another optional embodiment of the application, the server end receives the request sent by the client, and the request carries the state marker information of each client, and the state marker information is used to indicate that the client applies for joining or exiting the training process. The metadata is a feature description of each enterprise input data, and the metadata is a data structure in the form of an array or a tensor, which records the features, size, latitude, attribute, generation time, type, shape and other information of each input data or original data of the client. The server end has data labels Y, and the labels Y are historical production records of the host factory or are obtained by manually labeling historical production records and other original data. The original data includes inventory, sales, customer data, accessory data and other samples, and in a preferred embodiment, the original data can be all data related to the actual automobile production. The server end or the host factory can negotiate with the client to aggregate the data according to a time period t to obtain an input data set. Since the generation time of the original data is uncertain, data within a certain time period can be collected. Preferably, the time period t is, for example, 7 days, indicating that the aggregation is performed in weeks, or the aggregation can be performed every day, and the specific size of t can be set by the host factory, the server, the user or the client according to actual needs. The intermediate data obtained by processing the original data is used to construct an input data set, and the input data set can be stored in a database on the client. The input data can be composed of the features and samples of the data, represented by X i The tensor can be a vector, a matrix or a high-dimensional data structure.

[0073] Step S3: the server groups the clients and generates the split neural network. The server generates a sub neural network for each client according to the metadata of the client, preferably, the sub neural network is implemented by using a multi-layer perceptron or a Transformer or a recurrent neural network, and sends the sub neural network to the corresponding client; meanwhile, a plurality of client feature fusion neural networks and a feature fusion network located at the server are generated, the feature fusion network includes a plurality of group feature fusion sub-networks and a global fusion sub-network, so that the sub neural network, the client feature fusion neural network and the feature fusion network jointly constitute the split neural network. The input layer of the sub neural network of each client is generated according to the metadata of each client, the number of neurons is equal to the dimension of the input data, and the dimension is the number of features. The input layer of the sub neural network generated according to the metadata of each client is different, and any data of each corresponding client can be input. The output layer of all the sub neural networks has the same number of neurons, preferably, the output layer of all the sub neural networks has R neurons, R is a positive integer, and a fixed value can be set by a person skilled in the art according to the training situation. The sub neural network extracts features from the input data and sends the extracted features to the client feature fusion neural network. The client feature fusion neural network extracts features by using a multi-layer perceptron or a Transformer, the second last layer or the third layer obtains a first intermediate feature, and the last layer obtains a predicted value of a label. A first loss function is calculated according to the predicted value of the label and a multiplex label Y, and the extracted first intermediate feature is output to the group feature fusion neural network. The group feature fusion sub-network is dynamically generated according to the grouping of the clients and the number of clients in the group. The server can increase or delete neurons corresponding to the changed clients in the input layer of the group feature fusion sub-network according to the change of the clients in the group. The neurons added in the input layer are fully connected with the neurons in the adjacent layer (i.e. the second layer of the group feature fusion sub-network), and the remaining part of the group feature fusion sub-network remains unchanged. The added neurons can join the training process. The input layer of the group feature fusion sub-network fuses a plurality of first intermediate features, extracts features by using a multi-layer perceptron or a Transformer, and outputs a second intermediate feature through the output layer. The input layer of the global fusion sub-network is dynamically generated according to the grouping. The server can increase or delete neurons corresponding to the changed groups in the input layer of the global fusion sub-network. The input layer of the global feature fusion sub-network fuses a plurality of second intermediate features by using a multi-layer perceptron or a Transformer, and extracts high-order interactions between the second intermediate features between groups. In an optional embodiment of the present application, all the sub neural networks, client feature fusion neural networks, group feature fusion sub-networks and global fusion sub-networks are implemented by using a multi-layer perceptron.In an optional embodiment of the present application, all sub neural networks, client feature fusion neural networks, and group feature fusion sub networks are implemented using multi-layer perceptron, and all global fusion sub networks use Transformer. In an optional embodiment of the present application, the global fusion sub network can be built in the same mode as the group feature fusion sub network. The global fusion sub network is dynamically generated according to the grouping of the clients, and the server can add or delete neurons corresponding to the changed groups in the input layer of the global fusion sub network according to the change in the number of groups. The added neurons in the input layer are fully connected with the neurons in the next adjacent layer (i.e. the second layer of the global fusion sub network), and the rest of the global fusion sub network remains unchanged. The input layer of the global fusion sub network uses a convolution block to fuse multiple second intermediate features, uses one or two residual modules for feature extraction, and outputs a prediction value and a second loss function through the output layer. In an optional embodiment of the present application, each client generates a corresponding sub neural network according to its own metadata, without the need for the server to generate the sub neural network. In an optional embodiment, there is a client running in the server, and the server establishes a sub model M0 for the client running in the server, which is responsible for processing the local feature data of the server. At the same time, a sub model M is established for each client respectively. i (i = 1, 2, …, N); the server sends the client sub model M i to the corresponding client respectively. In the present application, for the sake of brevity, the client running on the server and the client not running on the server are handled together, but those skilled in the art should understand that the client in the present application can be partially or entirely running in the server. In an optional embodiment of the present application, the N clients are divided into M groups, and M group feature fusion sub networks, one global fusion sub network, and one prediction data multiplexing label Y are generated according to the grouping. The M group feature fusion sub networks correspond to the M groups one by one, and each group contains a number of clients and a client feature fusion neural network corresponding to the client. k ∈ [1, M]. k is a positive integer, the value of k corresponds to different groups one by one, and M is the number of groups. Preferably, clients of the same type are grouped into a group

[0074] In an optional embodiment of the present application, the present application provides a collaborative training method based on longitudinal federated learning, which comprises model training: first training a sub-neural network model of a client and a client feature fusion neural network on a server side, and then training a feature fusion network model on the server side. Each client first trains its own sub-neural network model, and for each group, a plurality of client feature fusion neural networks and a feature fusion network are trained on the server side, wherein the feature fusion network comprises a plurality of group feature fusion sub-networks and a global fusion sub-network; in the case of grouping a plurality of clients, each group of clients corresponds to a group feature fusion sub-network; the group feature fusion sub-network fuses the first intermediate features output by the client feature fusion neural networks in the group.

[0075] Step S4: Each client first trains a sub-model M i , which is a model composed of a sub-neural network running in the sub-neural network training and / or generating module of the client. Each client takes the original data X i stored in its own database as input data of the sub-model, and performs forward propagation along the sub-neural network to obtain a feature vector and sends it to the server side. The client feature fusion neural network on the server side uses the feature vector sent by the client to calculate the predicted value Y i (r) of the prediction data multiplexing label. The server side uses the prediction data multiplexing label Y to calculate the first loss function Based on the first loss function, at least one of the sub-neural network and the client feature fusion neural network is adjusted. Step S4 is iteratively executed, the first loss function is updated each time, and the iteration is stopped when the sub-neural network and the client feature fusion neural network converge, or the number of iterations reaches an iteration threshold, or the iteration time reaches a pre-set calculation time threshold, wherein r represents the number of iterations, and the predicted value of the label is generated once in each iteration process, which is a temporary intermediate value, and the last label prediction value is the predicted value of the label generated by the network when the iteration is stopped. Preferably, the maximum number of iterations r can be set by a person skilled in the art according to actual needs, for example, the maximum number of iterations r can be selected from 3 to 100000. In an optional embodiment, the gradient is calculated based on the first loss function and backpropagation is performed, and at least one of the sub-neural network and the client feature fusion neural network is adjusted, and the process is iterated according to step S4. Optionally, the loss L iThe encrypted data is then sent to the client. In another optional embodiment, the last layer of the client sub neural network receives the loss After that, the gradient is calculated and backpropagation is performed, and the process is iterated according to step S4, and a threshold value of the number of iterations is set, preferably rmax is 20 rounds. The sub model M i After the training is completed, the client C i uses M i The feature vectors of all samples (or input data) are calculated and sent to the server. The client feature fusion neural network of the server calculates the predicted value of the label according to the feature vector. In an optional embodiment of the present application, the server receives the output value of the sub neural network and inputs the output value into the client feature fusion neural network, and then determines the first intermediate feature fusion function value (also called the predicted value of the label in the present application) and the first loss function through the client feature fusion neural network and the predicted data multiplexing label Y. Based on the backpropagation of the first loss function, at least one of each sub neural network and the client feature fusion neural network is adjusted, and the process is repeated until the sub neural network and the client feature fusion neural network converge, and the step is stopped. The client feature fusion neural network outputs the first intermediate feature and sends it to the corresponding group feature fusion sub network; the first intermediate feature can be output by the network layer of the client feature fusion neural network that extracts the feature, such as the penultimate layer of neural network or the antepenultimate layer of neural network, and those skilled in the art can select the network layer that can extract the specific feature according to the actual needs, which is not limited in the present application. Figure 3 As shown in the figure, for example, the penultimate layer of neural network or the antepenultimate layer of neural network, those skilled in the art can select the network layer that can extract the specific feature according to the actual needs, which is not limited in the present application.

[0076] By multiplexing the label in the client feature fusion neural network, the asynchronous processing of the split neural network is realized asynchronously through the label. On the one hand, the client sub neural network and the client feature fusion neural network do not affect the calculation of the feature fusion network (including multiple group feature fusion sub networks and one global fusion sub network) when iteratively calculating, and different sub networks of the entire network can not affect each other when independently performing iteration, realizing the asynchronous processing of the front and back of the network. On the other hand, the second loss function does not have to be transmitted to each client, reducing the calculation on the node and saving the operation time, and accelerating the network convergence speed.

[0077] In an optional embodiment, the present application provides a collaborative training method based on longitudinal federated learning. For each client, the sub neural network runs in the sub neural network training and / or generation module, and in each round of iterative training, the sub neural network training and / or generation module obtains the original data or input data X i of the current iteration from the database of the client to which it belongs. iThe original data can be obtained according to the rules set by the person skilled in the art, and the input data X i The intermediate data obtained after processing the original data can also be obtained, and one round of inference calculation and training is completed on each sub neural network. The output of the sub neural network is transmitted to the client feature fusion neural network on the server side through the communication module. The server side calculates the first intermediate feature fusion function value F1(X i ) and the first loss function L1(X i ) corresponding to each client through the client feature fusion neural network and the shared prediction data label Y, and performs back propagation to update the parameters P j of the sub network model. Wherein j is a positive integer representing the number corresponding to each neuron in each sub network. The server side and the client adjust at least one of each sub neural network and the client feature fusion neural network based on the back propagation of the first loss function. The iteration is continuously iterated until the sub neural network and the client feature fusion neural network converge or reach the maximum iteration number or the iteration time exceeds the preset calculation time threshold, and the iteration is stopped. The client feature fusion neural network outputs the first intermediate feature and sends it to the corresponding group feature fusion sub network. The first intermediate feature is output by the network layer of the client feature fusion neural network for extracting data features, preferably the network layer of the client feature fusion neural network for extracting data features is the second to last layer or the third to last layer.

[0078] Step S5: the server side generates the second intermediate feature by fusing at least one first intermediate feature in the group through the group feature fusion neural network, and establishes a group feature fusion sub network for each group on the server side for fusing the output of the client feature fusion neural network corresponding to each client in the group. When all the client feature fusion neural networks in a group are trained, the group feature fusion sub network of each group is trained so as to fuse the first intermediate feature output by the client feature fusion neural network corresponding to the client in the group to obtain the second intermediate feature, and send the second intermediate feature to the global fusion sub network. In an optional embodiment of the present application, the server side adds or deletes neurons in the input layer of the group feature fusion sub network for the dynamically added or exited clients in the group according to the state marker information carried in the request sent by the client. The neurons are used to receive the first intermediate feature output by the client feature fusion neural network. The group feature fusion sub network receives the output values of the client sub neural network corresponding to all clients in the group and fuses them to obtain the second intermediate feature, and inputs the second intermediate feature into the global fusion sub network.

[0079] In an optional embodiment of the present application, the present application provides a collaborative training method based on longitudinal federated learning, wherein the step S5 of the method further comprises: the server side pre-designs a calculation time threshold, and judges whether the time spent by the client corresponding to the sub neural network and the client feature fusion neural network in converging exceeds the calculation time threshold, and if so, the server side deletes the neurons corresponding to the client in the input layer of the group feature fusion sub network, and sends relevant indication information of deleting the neurons to the client, and if the deleted client re-sends the request, the server re-adds the neurons in the input layer of the group feature fusion sub network for the client, and at the same time, the server side dynamically generates a client feature fusion neural network and a sub neural network with smaller generation time for the corresponding client, or the server side dynamically generates a client feature fusion neural network with less generation time for the corresponding client, and the client generates a sub neural network with less generation time according to its own metadata information. If the deleted client re-sends the request, the request carries the metadata and state marker information of the client, the server re-adds the neurons in the input layer of the group feature fusion neural network for the client according to the state marker information in the request, and when the server side dynamically generates a client feature fusion neural network and a sub neural network for the corresponding client, at least one of the client feature fusion neural network and the sub neural network is compressed, and the sub neural network after compression is used to update the existing sub neural network of the corresponding client, so as to reduce the time spent by the client in the training process, the client feature fusion neural network receives the features sent by the newly added sub network, the newly added client feature fusion neural network receives the features sent by the newly added sub network and generates first intermediate features after fusion processing, and sends the first intermediate features to the corresponding newly added neurons, and the newly added neurons can join the training process; or, if the deleted client re-sends the request, the request carries the state marker information of the client, the server re-adds the neurons in the input layer of the group feature fusion neural network for the client according to the state marker information in the request, the server side dynamically generates a compressed client feature fusion neural network for the corresponding client, and / or the client generates a compressed sub neural network according to its own metadata, the client feature fusion neural network receives the features sent by the newly added sub network, the newly added client feature fusion neural network receives the features sent by the newly added sub network and generates first intermediate features after fusion processing, and sends the first intermediate features to the corresponding newly added neurons, and the newly added neurons can join the training process.

[0080] Through the above steps, if there are problems such as expiration of the upstream and downstream cooperation period, change of commercial interests, client crash, etc., they can be well adapted, and the dynamic exit or entry of the client is realized, and the client and the network can be dynamically expanded, so that the network is more compatible and robust. Some clients that require a lot of calculation time are deleted, so that clients within the calculation time threshold do not have to wait for clients that take a long time to calculate, saving calculation time, and for clients whose data are important, they can join the task in time by re-sending the request. In addition, the network size can also be dynamically adjusted according to the situation, such as reducing the number of layers of the neural network or reducing the number of neurons in a certain layer, etc., so as to control the time required for each client to operate according to actual needs.

[0081] Step 6: The server determines the predicted value and the second loss function through the global fusion sub-network and the prediction data multiplexing label Y, and adjusts at least one of the respective sub-neural networks, the client feature fusion neural network, the group feature fusion sub-network and the global fusion sub-network based on the back propagation of the second loss function. Steps 4 to 6 are executed in a loop until the split neural network converges or the maximum number of iterations is reached or the time spent exceeds the maximum time threshold, after which steps four to six are stopped. The maximum number of iterations or the maximum time threshold can be preset in advance by a person skilled in the art according to actual needs.

[0082] Step 7: The final predicted value is output by the global fusion sub-network of the server through the input of the original data or the intermediate data obtained by processing the original data by the sub-neural network of the client. Preferably, for example, the final predicted value represents the number of cars that should be produced by the host factory next month. In an optional embodiment, during the prediction process, if a client sends a request to join or exit, for the exiting client, the server deletes the neurons in the input layer of the group feature fusion neural network corresponding to the exiting client, and notifies the exiting client of the information about the deletion of the relevant neurons. For the joining client, the server re-adds neurons in the input layer of the group feature fusion neural network for the client, the added neurons copy the parameters of the neighboring neurons in the input layer of the group feature fusion neural network, and the neurons are fully connected with the neurons of the next layer network. The server dynamically generates the client feature fusion neural network and the sub-neural network corresponding to the client, and sends the sub-neural network to the added client. The client feature fusion neural network receives the features sent by the added sub-network, and the added client feature fusion neural network receives the features sent by the added sub-network and generates the first intermediate features after fusion processing. The first intermediate features are sent to the corresponding added neurons. Only the sub-neural network and the corresponding client feature fusion neural network of the added client need to be re-trained using the prediction data multiplexing label Y to obtain a new split network.

[0083] The upstream and downstream enterprises in the supply chain do not need to share their original data, and the influence of the original data of the upstream and downstream enterprises is considered to form data collaborative training analysis of each enterprise, so that the final value prediction is more accurate, and the information island dilemma is broken.

[0084] In an optional embodiment of the present application, the present application provides a collaborative training method based on longitudinal federated learning, in step three of the method, the input layer of the sub neural network is dynamically generated according to the metadata to be input by the corresponding client, and the output layer of all the sub neural networks has the same number of neurons, which is preset by the server side or negotiated among the multiple clients.

[0085] In an optional embodiment of the present application, the present application provides a collaborative training method based on longitudinal federated learning, in the method, part or all of the clients can run in the server.

[0086] In an optional embodiment of the present application, the present application provides a collaborative training method based on longitudinal federated learning, in the method, the metadata records the data input by the client in the form of a tensor, including the characteristics, size, latitude, attribute, generation time, type and shape of the input data, and the metadata indication information includes the client flag, the metadata type to be input and the dimension.

[0087] In an optional embodiment of the present application, the present application further provides a collaborative training device based on longitudinal federated learning, the device comprises at least one processor and a memory in communication connection with the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of the embodiments of the present application.

[0088] In an optional embodiment of the present application, the present application further provides a computer readable storage medium, the computer readable storage medium stores a program, when the program is executed by a multi-core processor, the multi-core processor executes the method according to any one of the embodiments of the present application.

[0089] Embodiment 2

[0090] According to the embodiments of the present application, a supply chain demand prediction method based on collaborative training of longitudinal federated learning is provided, the longitudinal federated learning idea refers to federated learning divided by features, and is suitable for a federated learning scene composed of participants with the same sample space and different feature spaces.

[0091] Figure 4A supply chain demand prediction method based on longitudinal federated learning according to an embodiment of the present application is shown in FIG. 1. Figure 4 The method comprises the following steps:

[0092] S201, obtaining feature data of upstream and downstream enterprises in a supply chain, the feature data being original data, which is input through a sub-neural network of a client;

[0093] According to the idea of longitudinal federated learning, the feature data of each enterprise in the supply chain is combined to train a federated model. The upstream supplier provides original data such as order, inventory, production status, and shipment quantity, the midstream distributor provides original data such as order and inventory, and the downstream retailer provides original data such as sales and inventory. The original data is sampled and valued at a certain time interval, for example, a time interval of one week.

[0094] The manufacturing plant enterprise is a client running on a server S of federated learning, and a client model C s is maintained at the same time. Preferably, the server S can be owned by the manufacturing plant enterprise, and preferably, the server S can also be a server of a third-party company rented by the manufacturing plant enterprise. The distributors, retailers, and other suppliers at all levels are clients C i (i = 1, 2, …, n), where n represents the number of clients. Preferably, C s may be one of the clients C i . Preferably, C s may be a client independent of C i . The local data of each client is stored in a local database through a data processing module.

[0095] S202, establishment and distribution of a split neural network model;

[0096] The server S establishes a sub-model M i for each enterprise client C i , and sends the sub-neural network M i to the corresponding client through a communication module. In an optional embodiment, the sub-neural network M i of all clients is established by the clients themselves and the output values of the sub-neural network can be sent to the server. In an optional embodiment, the sub-neural network of part of the enterprise clients is established by the enterprise clients themselves and the output values of the network can be sent to the server, and the sub-neural network of the other part of the enterprise clients is established by the server S for each enterprise client C i and sent to the corresponding client through the communication module. i

[0097] ​Then, clients are grouped into different types of groups according to different supply levels (e.g., retailer group G). d Distributor Group G a Manufacturing Plant Group G m The server-side S represents G. d G a and G m A group feature fusion subnetwork M is established. G，d M G，a M G，m The outputs of the client feature fusion neural networks corresponding to the sub-neural networks of each client in the group are used to fuse the outputs of the client feature fusion neural networks. Each group feature fusion sub-network, as well as the clients in the group and the client feature fusion neural networks corresponding to the clients in the group, belong to the same group. The group contains the sub-neural networks located on the client side, as well as multiple client feature fusion neural networks and multiple group feature fusion sub-networks located on the server side. All the neural networks in a group constitute a local fusion model.

[0098] Then, establish a globally fused subnetwork M on the manufacturing plant server side. global It is used to fuse the outputs of each local fusion model. All sub-neural networks, client feature fusion neural networks, group feature fusion sub-networks and global fusion sub-networks together form the global model M.

[0099] S203, Model Training;

[0100] Based on the demand forecasting task, various clients such as retailers, distributors, and manufacturers combine dynamic encryption algorithms and utilize the collaborative training method of vertical federated learning disclosed in this invention to collaboratively train the split network model established in step S202.

[0101] like Figure 5 As shown, the training process can be divided into the following four steps:

[0102] S203_1: Retailers, distributors, and manufacturers' clients use the raw data obtained in step S201 to optimize the sub-neural network M. i Training is performed. Input the original data X. i Along M i Forward propagation yields sub-feature vectors And The data is sent to the server. The sub-feature vector is the output feature of the client's sub-neural network. The server utilizes... Calculate the predicted value Y of product demand. i After r iterations, the sub-feature vectors can be obtained. and predicted values Preferably, such as Figure 3 As shown, the sub-feature vector from the sub neural network M i the second last layer or the third last layer.

[0103] S203_2: obtain the predicted value Y of each client's demand in each round of training i After that, the server side calculates the first loss L1 i using the predicted data multiplexing label Y through the client feature fusion neural network, and then uses a homomorphic encryption algorithm to encrypt L1 i After encryption, it is sent to the client, and the client receives L1 i After that, the gradient is calculated and back propagation is performed to adjust the sub neural network of the client. In another embodiment, based on L1 i The gradient is calculated and sequentially back propagated to the corresponding client feature fusion neural network and the client's sub neural network to adjust at least one of the client feature fusion neural network and the sub neural network.

[0104] The server side pre-designs a calculation time threshold. If the time spent by the client's corresponding sub neural network and the client feature fusion neural network to converge exceeds the calculation time threshold, the server side deletes the neurons in the input layer of the group feature fusion neural network corresponding to the client, and sends relevant indication information of deleting the neurons to the client.

[0105] If the deleted client re-sends the request or a new client re-sends the request, the request carries state marker information of the client applying to join. The server re-adds neurons in the input layer of the group feature fusion neural network for the client. When the server side dynamically generates the client feature fusion neural network and the sub neural network for the corresponding client, at least one of the client feature fusion neural network and the sub neural network is compressed, and the compressed sub neural network is used to update the existing sub neural network of the corresponding client. The client feature fusion neural network receives the features sent by the newly added sub network, and the newly added client feature fusion neural network receives the features sent by the newly added sub network and generates first intermediate features after fusion processing. The first intermediate features are sent to the corresponding newly added neurons, which can join the training process or,

[0106] If the client re-sends the request or adds a client re-sending request, the request carries the state marker information of the client applying to join, the server dynamically generates a compressed client feature fusion neural network for the corresponding client and / or the client generates a compressed sub-neural network according to its own metadata, the client feature fusion neural network receives the features sent by the added sub-network, the added client feature fusion neural network receives the features sent by the added sub-network and generates first intermediate features after fusion processing, and sends the first intermediate features to the corresponding added neurons, which can join the training process.

[0107] S203_3: When the client sub-neural network and the client feature fusion neural network in one group are trained, the client feature fusion neural network outputs first intermediate features and sends them to the group feature fusion sub-network, then the training of the corresponding local fusion model of the group begins, and the first intermediate features of the clients in the group are fused to determine the second intermediate features

[0108] S203_4: All local fusion models and global fusion sub-networks jointly constitute a global model M, and the second intermediate features are input into the global fusion sub-network to obtain a global demand prediction value Z2, where 1 is the iteration number of the second intermediate features, and the value of k corresponds to different groups one by one, i.e., k corresponds to the number of groups, and the server calculates the second loss L2 using the prediction data multiplexing label Y i , adjusts at least one of the sub-neural network, the client feature fusion neural network and the feature fusion neural network based on the second loss function, and repeatedly iterates and adjusts the steps of the global model to obtain the second intermediate features where q is the iteration number of the second intermediate features, and the value of k corresponds to different groups one by one, i.e., k corresponds to the number of groups, and the sub-neural network, the client feature fusion neural network and the feature fusion neural network are adjusted according to the second loss function. Preferably, the gradient is calculated by the second loss function, and the multiple group feature fusion sub-networks and one global fusion sub-network are adjusted by back propagation.

[0109] S204, demand prediction is performed according to the global model M.

[0110] The sub-neural networks, client feature fusion neural networks, group feature fusion sub-networks and global fusion sub-networks jointly constitute a global model M. Retailers, distributors and manufacturers input local raw data into the global model M and perform a forward propagation to obtain a demand prediction value In the process of prediction, if a client sends a request to join or quit, for the quitting client, the server deletes the neurons corresponding to the quitting client in the input layer of the group feature fusion neural network, and notifies the quitting client of the information of the deleted neurons. For the joining client, the server re-adds neurons in the input layer of the group feature fusion neural network for the client, the added neurons copy the parameters of the neighbor neurons in the input layer of the group feature fusion neural network, the neurons are fully connected with the neurons of the next layer network, the server dynamically generates a client feature fusion neural network and a sub neural network for the corresponding client again, and sends the sub neural network to the added client, the client feature fusion neural network receives the features sent by the added sub network, the added client feature fusion neural network receives the features sent by the added sub network and generates first intermediate features after fusion processing, and sends the first intermediate features to the corresponding added neurons, and only the sub neural network of the added client and the corresponding client feature fusion neural network need to be retrained by using the prediction data multiplexing label Y to obtain a new split network.

[0111] Through the above steps, a longitudinal federated learning model is established based on multi-party private data to perform demand prediction. The channel for collaborative planning, prediction and replenishment from an upstream manufacturer to a downstream retailer is opened, and terminal sales data is timely and accurately transmitted to the manufacturer, so that accurate demand prediction is realized, and problems such as large inventory backlog, poor capital turnover, extended delivery period and logistics in the development process of the supply chain are solved, and the competitiveness of the enterprise is improved.

[0112] Embodiment 3

[0113] A car is composed of thousands of parts, and different parts are produced by different enterprises. The part manufacturers have complete part attribute data, but the prediction of carbon emissions based on this is not accurate enough because it does not take into account the carbon emissions generated by the assembly of parts. According to the idea of longitudinal federated learning, the feature data of the part manufacturers involved in the whole vehicle is combined to train a federated model.

[0114] According to an embodiment of the present application, a whole vehicle carbon emission prediction method based on collaborative training of longitudinal federated learning is provided. As shown in Figure 6 the method comprises the following steps:

[0115] S301, obtaining part data and carbon emission data thereof, the part data and the carbon emission data thereof constituting original data;

[0116] The whole vehicle enterprise is taken as a server S of federated learning, and a client model C s Various part manufacturers are taken as clients C of federated learning i(i = 1, 2, …, n), where n represents the number of clients, preferably, C s may be a function of C i for one client, where s e [1, n], n is a positive integer greater than 1. Preferably, C s may be a function of C i for one client. The raw data of each client is stored in the database of the client by the data processing module.

[0117] S302, establishment and distribution of the split neural network model;

[0118] The server S establishes a sub-model M i for each component client C i , and sends the sub-model M i to the corresponding client through the communication module. In an optional embodiment, the sub-neural network M i of all clients is established by the clients themselves and the output values of the network can be sent to the server. In an optional embodiment, the sub-neural network M i of part of the enterprise clients is established by the enterprise clients themselves and the output values of the network can be sent to the server, and the sub-neural network M i of the other part of the enterprise clients is established by the server S for each enterprise client C i and sent to the corresponding client through the communication module. i

[0119] Then the clients are divided into different groups G k (k = 1, 2, …, m), where G represents a group, k is the number of the group, and m represents the number of groups. The server S establishes a local fusion model M k for each group G G，k to fuse the outputs of the client feature fusion neural networks corresponding to the sub-neural networks of the clients in the group. Each group feature fusion sub-network, the clients in the group, the sub-neural networks of the clients, and the client feature fusion neural networks corresponding to the clients in the group belong to one group. The group contains the sub-neural networks located in the clients and the multiple client feature fusion neural networks and multiple group feature fusion sub-networks located in the server. All the neural networks in one group constitute a local fusion model.

[0120] Then a global fusion sub-network M global is established at the server of the vehicle enterprise to fuse the outputs of the local fusion models. All the sub-neural networks, client feature fusion neural networks, group feature fusion sub-networks, and global fusion sub-networks together form a global model M.

[0121] ​S303, Model Training;

[0122] Based on the carbon emission prediction task, each component client uses the collaborative training method of vertical federated learning disclosed in this invention to collaboratively train the vertical federated learning model established in step S302. For example... Figure 7 As shown, the training process can be divided into the following four steps:

[0123] S303_1: Each component client uses the raw data obtained in step S301 to process the sub-neural network M. i Training is performed. Input the original data X. i Along M i Forward propagation yields sub-feature vectors And Send to the server. The server uses... Calculate the predicted value of carbon emissions Y i After r iterations, the sub-feature vectors can be obtained. and carbon emission forecasts Preferably, such as Figure 3 As shown, the sub-feature vector From sub-neural network M i Obtained from the second to last or third to last level.

[0124] S303_2: The predicted carbon emissions Y for each component client is obtained in each round of training. i Subsequently, the server calculates the first loss L1 by fusing client features with a neural network and reusing the label Y using historical carbon emission prediction data. i Then, use a homomorphic encryption algorithm on L1 i It is encrypted before being sent to the client. The client receives L. i The gradient is then calculated and backpropagation is performed to adjust the sub-neural network of the client. In another embodiment, based on L1... i The gradient is calculated and propagated backwards sequentially to the corresponding client feature fusion neural network and the client sub-neural network, thereby adjusting the client feature fusion neural network and the sub-neural network.

[0125] S303_3: When at least one of the client sub-neural network and the client feature fusion neural network within a group has been trained, the client feature fusion neural network outputs a first intermediate feature. The data is then sent to the group feature fusion subnetwork, which then begins training the corresponding local fusion model for that group, fusing the first intermediate features from the clients within the group. Identify the second intermediate feature

[0126] S303_4: all local fusion models and global fusion sub-networks jointly constitute a global model M, and the second intermediate feature The global fusion sub-network is input to obtain a global carbon emission prediction value Z2, and the server end uses the prediction data to multiplex the label Y to calculate a second loss L2 i , based on the second loss function, at least one of the sub neural network, the client feature fusion neural network and the feature fusion neural network is adjusted. The step of continuously circulating and adjusting the global model is obtained Wherein, q is the iteration number of the second intermediate feature, the value of k corresponds to different groups one by one, that is, k corresponds to the number of groups of the grouping, and at least one of the sub neural network, the client feature fusion neural network and the feature fusion neural network is adjusted according to the second loss function. Preferably, the gradient is calculated by the second loss function, and the multiple group feature fusion sub-networks and the global fusion sub-network are adjusted by back propagation.

[0127] S304, carbon emission prediction is carried out according to the global model M.

[0128] Each sub neural network, client feature fusion neural network, group feature fusion sub-network and global fusion sub-network jointly constitutes a global model M. Each component client inputs the local component data into the global model M and carries out a forward propagation to obtain a carbon emission prediction value In the process of prediction, if a client sends a request to join or quit, for the quitting client, the server deletes the neurons corresponding to the quitting client in the input layer of the group feature fusion neural network, and notifies the deleted client of the information of deleting the related neurons. For the joining client, the server re-adds the neurons in the input layer of the group feature fusion neural network for the client, the added neurons copy the parameters of the neighbor neurons in the input layer of the group feature fusion neural network, the neurons are fully connected with the neurons of the next layer network, the server end dynamically generates the client feature fusion neural network and the sub neural network for the corresponding client again and sends the sub neural network to the added client, the client feature fusion neural network receives the features sent by the added sub network, the added client feature fusion neural network receives the features sent by the added sub network and generates the first intermediate feature after fusion processing, and sends the first intermediate feature to the corresponding added neurons. Only by retraining the sub neural network and the corresponding client feature fusion neural network of the added client using the prediction data multiplexing label Y can the new split network be obtained.

[0129] Through the above steps, a vertical federated learning model is established based on multi-party private data to predict carbon emissions. The distributed training is carried out by combining the component data of the component manufacturers of each subsystem in the vehicle, so as to realize accurate carbon emission prediction.

[0130] It should be noted that the steps not described in detail in the embodiments can refer to the descriptions of the related steps in the embodiments shown in the drawings, which will not be described here. Figure 1

[0131] In the description of the present specification, the description of the terms "some possible embodiments", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiments or examples are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0132] Regarding the method flowchart of the embodiments of the present application, some operations are described as different steps executed in a certain order. Such flowcharts are illustrative rather than limiting. Some steps described herein can be grouped together and executed in a single operation, some steps can be divided into multiple sub-steps, and some steps can be executed in an order different from that shown herein. Each step shown in the flowchart can be implemented in any way by any circuit structure and / or tangible mechanism (for example, by software running on a computer device, hardware (for example, processor or chip implemented logic function), etc., and / or any combination thereof) in any manner.

[0133] Those skilled in the art will appreciate that embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] ​The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more functions specified in the flowchart block or blocks. Figure 1 one or more functions specified in the flowchart block or blocks.

[0135] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more functions specified in the flowchart block or blocks. Figure 1 one or more functions specified in the flowchart block or blocks. In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0136] Computer readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. In addition, although the operations of the method of the present application are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in that particular order, or that all of the illustrated operations must be performed to achieve the desired result. Additionally or alternatively, certain steps can be omitted, combined into a single step, and / or divided into multiple steps.

[0137] While the principles and spirit of the application have been described with reference to several specific embodiments, it is to be understood that the application is not limited to the specific embodiments disclosed, and that the division of the aspects is not meant to imply that features from these aspects cannot be combined to benefit, but is merely for ease of presentation. The application is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A collaborative training method based on longitudinal federated learning, characterized in that, the split neural network established includes a sub neural network located at each client and a plurality of client feature fusion neural networks and a feature fusion network located at the server side, wherein the feature fusion network includes a plurality of group feature fusion sub networks and a global fusion sub network, each group feature fusion sub network corresponding to a group of clients; the server side receives the output features of each sub neural network and inputs each of the output features into the corresponding client feature fusion neural network to determine a first loss function for each client feature fusion neural network through the output results of the client feature fusion neural network and the prediction data multiplexing label; and at least one of the sub neural network and the client feature fusion neural network is adjusted based on the first loss function; the server side receives the first intermediate features output by each adjusted client feature fusion neural network through the feature fusion network, wherein each group feature fusion sub network receives the first intermediate features output by the corresponding client feature fusion neural network of the clients in the group, and fuses the received first intermediate features to obtain the second intermediate features corresponding to each group feature fusion sub network; and the global fusion sub network receives the second intermediate features corresponding to the plurality of group feature fusion sub networks respectively to determine the output results of the feature fusion network; and a second loss function for the feature fusion network is determined based on the output results of the feature fusion network and the prediction data multiplexing label; and at least one of the sub neural network, the client feature fusion neural network and the feature fusion network is adjusted based on the second loss function.

2. The method of claim 1, characterized in that, in the case of grouping a plurality of clients, each group of clients corresponds to a group feature fusion sub network; for each of the plurality of group feature fusion sub networks, the first intermediate features output by the corresponding client feature fusion neural network of the clients in the group are received and fused to obtain the second intermediate features corresponding to each group feature fusion sub network; the output results of the global fusion sub network are determined based on the global fusion sub network receiving the second intermediate features corresponding to the plurality of group feature fusion sub networks respectively.

3. The method of claim 2, wherein, adjusting at least one of the sub neural network and the client feature fusion neural network based on the first loss function further includes: performing loop iteration through back propagation of the first loss function, adjusting at least one of the sub neural network and the client feature fusion neural network, until the sub neural network and the client feature fusion neural network converge, or the iteration reaches an iteration threshold, or the iteration time reaches a pre-set calculation time threshold, the client feature fusion neural network outputs the first intermediate features and sends them to the corresponding group feature fusion sub network.

4. The method of claim 2, wherein, The adjusting at least one of the sub neural network, the client feature fusion neural network, and the feature fusion network based on the second loss function further includes: The adjusting at least one of the sub neural network, the client feature fusion neural network, the group feature fusion sub network, and the global fusion sub network is performed through loop iteration by back propagation of the second loss function, until the iteration stops after the split neural network converges or after the iteration number reaches an iteration threshold value or after the iteration time reaches a pre-set calculation time threshold value, and the collaborative training of the vertical federated learning of the split neural network is completed.

5. The method of claim 4, wherein, The server-side global fusion sub network outputs a final prediction value through the sub neural network of the client inputting original data.

6. The method of claim 3, wherein, The server side pre-designs a calculation time threshold value, if the time spent by the iteration in adjusting the sub neural network and the client feature fusion neural network of the client exceeds the calculation time threshold value, the server side deletes the neurons in the input layer of the group feature fusion neural network corresponding to the client, and sends the related indication information of the deleted neurons to the client corresponding to the neurons; If the deleted client re-sends a request or a new client sends a request, the request carries the metadata and / or state marker information of the client, the server re-adds the neurons in the input layer of the group feature fusion neural network for the client, and when the server side dynamically generates the client feature fusion neural network and the sub neural network for the corresponding client, at least one of the client feature fusion neural network and the sub neural network is compressed, and the existing sub neural network of the corresponding client is updated with the compressed sub neural network; Or, If the deleted client re-sends the request or a new client sends a request, the request carries the metadata and / or state marker information of the client, the server side dynamically generates a compressed client feature fusion neural network for the corresponding client, and / or the client generates a compressed sub neural network according to its own metadata.

7. The method of claim 6, wherein, The sub neural network is generated according to the metadata of the client, so that the input layer of the sub neural network can input various original data of the client, and all output layers of the sub neural network have a preset number of neurons, which is pre-set by the server side or negotiated and set by multiple clients.

8. The method according to any one of claims 1 to 7, characterized in that, At least one of the clients runs in the server.

9. A device for collaborative training based on longitudinal federated learning, characterized in that, Comprises: At least one processor; And a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

10. A computer readable storage medium, the computer readable storage medium storing a program, when the program is executed by a multi-core processor, the multi-core processor executes the method of any one of claims 1-8.

Images