A dynamic construction method and device for splitting a learning neural network and a storage medium

Abstract

The application provides a dynamic construction method and device for splitting a learning neural network and a storage medium. A server end groups multiple clients, dynamically generates a global fusion neural network according to the grouping, and sends a request carrying metadata and state marker information of inputs of each client to the server end. The server end dynamically generates a sub neural network and a client feature fusion neural network for the corresponding client according to the metadata, sends the sub neural network to the corresponding client, dynamically generates a corresponding group feature fusion neural network for each group according to the number of clients in each group and the state marker information in the request, and makes the sub neural network, the client feature fusion neural network, the group feature fusion neural network and the global fusion neural network jointly constitute a split neural network. The calculation time of the sub neural network is controlled, the clients can be dynamically expanded, the robustness is higher, and accurate prediction of a prediction target is realized.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of machine learning, and particularly relates to a collaborative training method and device based on longitudinal federated learning and a storage medium. BACKGROUND

[0002] This section is intended to provide background or context to the embodiments of the application recited in the claims. The description herein does not constitute admission that the prior publication, square, or subject matter described herein and / or the material contained therein is or was prior art to the claims at issue.

[0003] In the supply chain, the enterprise manufacturer produces products in the factory and sends them to the distributor, who sends them to the downstream retailer. The manufacturer has complete shipment data, but fails to consider the actual sales and inventory of downstream distributors and retailers when predicting future demand based on this data, resulting in often inaccurate predictions. For example, predicting the number of cars produced or demanded by the enterprise next month using only internal data: sales data, inventory data, historical production data, and parts data. However, it is found that the prediction accuracy is not high enough when using only internal data. Combining data from upstream and downstream of the supply chain, such as the data from distributors, second-tier distributors, and third-tier distributors to predict production, the data from each enterprise in the supply chain can be used as an influencing feature, and more data sources can be considered for prediction to improve prediction accuracy.

[0004] In the first aspect, due to confidentiality obligations, information security, business competition, and other reasons, enterprises in the supply chain are often unwilling to share their own data, resulting in low data sharing among upstream, midstream, and downstream enterprises, and a large amount of data cannot be fully utilized, resulting in a data island problem. Therefore, how to form a collaborative training analysis method for enterprise data is an urgent problem for enterprises in the value chain.

[0005] In the second aspect, due to the different computing capabilities of different clients, when a faster client transmits data to the server, the server needs to wait for the remaining clients to transmit data to the server. After the server waits for data synchronization, the server can continue to perform calculations, and asynchronous calculations cannot be achieved. If the number of clients is large, or some clients are very slow, it will seriously affect the entire network training and prediction time.

[0006] In the third aspect, in practice, due to various reasons such as cooperation period, business interest changes, and client crashes, clients often need to dynamically exit or enter the network, but the existing network does not allow clients to dynamically exit or join. Once one or more clients exit, the server network cannot synchronize the relevant data, resulting in data synchronization problems and making the network unusable. If a client wants to join, but the server and client network in the prior art cannot be expanded, and cannot be compatible with the newly added client.

[0007] Fourthly, the prior art often requires input data to be aligned, which will result in additional calculation, and if the data is not aligned or the alignment is wrong, the calculation cannot be performed

[0008] Fifthly, the network of the prior art is fixed once it is built, and for some occasions where the network size is not suitable, a large amount of calculation time is required, and the network size cannot be dynamically adjusted according to the actual situation.

[0009] Sixthly, in the prior art, the network is a whole, and when the front part of the network is performing calculation, the back end of the network can only wait, and different parts of a deep network cannot be asynchronously run.

[0010] Seventhly, in the prior art, the split neural network is calculated using labels at the end of the network, and then back propagation is performed, the partial derivative of the loss function with respect to the neuron parameters is calculated to obtain error information, the error information is transmitted from the server to all clients, and each client needs to perform internal back propagation after receiving the error information, and the next forward propagation cannot be performed until all clients complete the back propagation, which results in a long message synchronization time and a large number of node transmissions, and a large amount of time is spent, resulting in slow convergence of the split neural network. SUMMARY

[0011] In view of the above problems in the prior art, a dynamic construction method of a split learning neural network and a supply chain demand prediction method and device based on vertical federated learning collaborative training and a computer readable storage medium are provided, which can partially or completely solve the above problems.

[0012] The present application provides the following solutions.

[0013] In an optional embodiment, the present application provides a dynamic construction method of a split learning neural network,

[0014] The server divides a plurality of clients into a plurality of groups, and dynamically generates a global fusion neural network according to the grouping;

[0015] receive requests sent by a plurality of clients, each of the requests sent by each of the clients carrying metadata and state marker information of each of the clients, the metadata being used to indicate information of input data of the client, and the state marker information being used to indicate that the client applies to join or quit a training process, the training process referring to a process of constructing and training the split learning neural network at the server and the clients;

[0016] dynamically generating a corresponding client feature fusion neural network for the corresponding client according to the metadata in the request, and dynamically generating a corresponding group feature fusion neural network for each group according to the number of clients in each group and the state marking information in the request; or

[0017] dynamically generating a corresponding client feature fusion neural network for the corresponding client according to the metadata in the request, and dynamically generating a corresponding group feature fusion neural network for each group according to the number of clients in each group and the state marking information in the request; or

[0018] dynamically generating a corresponding client feature fusion neural network for the corresponding client according to the metadata in the request, and dynamically generating a corresponding group feature fusion neural network for each group according to the number of clients in each group and the state marking information in the request; or

[0019] In an optional embodiment, the application provides a dynamic construction method of a split learning neural network, and the dynamic generation of the corresponding sub-neural network further comprises: dynamically generating the number of input layer neurons of the sub-neural network according to the metadata of each client, wherein the number of input layer neurons is equal to the dimension of the input data of the corresponding client, and dynamically generating the same number of neurons for the output layer of all the sub-neural networks, wherein the number of output layer neurons is preset in advance by the server side or is set by negotiation between the plurality of clients.

[0020] In an optional embodiment, the application provides that the server side receives the output value of the sub-neural network and inputs the output value into the client feature fusion neural network, and then determines the first intermediate feature fusion function value and the first loss function through the client feature fusion neural network and the prediction data multiplexing label Y, and the server side and the client adjust at least one of each sub-neural network and the client feature fusion neural network based on the back propagation of the first loss function, iteratively perform the step of adjusting the neural network based on the back propagation of the first loss function until the sub-neural network and / or the client feature fusion neural network converges, and then stop executing the iteration, and the client feature fusion neural network outputs the first intermediate feature and sends it to the corresponding group feature fusion neural network;

[0021] In an optional embodiment, the application provides a dynamic construction method of a split learning neural network, and the dynamic generation of a corresponding group feature fusion neural network for each group according to the number of clients in each group and the state marking information in the request further comprises:

[0022] The server end adds or deletes neurons in the input layer of the group feature fusion neural network for dynamically joining or leaving the group of clients according to the state marker information, the newly added neurons can join the training process in real time, the training process refers to the process of building and training the split learning neural network at the server end and the client end, the neurons are used to receive the first intermediate features output by the client feature fusion neural network, the group feature fusion neural network receives the output values of the sub neural networks corresponding to all clients in the group and fuses to obtain second intermediate features, and inputs the second intermediate features into the global fusion neural network.

[0023] In an optional embodiment, the application provides a dynamic construction method of a split learning neural network, the server end determines the output result of the global fusion neural network based on a plurality of second intermediate features through the global fusion neural network, and determines the prediction value and the second loss function based on the output result and the prediction data multiplexing label Y, the server end and the client end adjust at least one of each sub neural network, the client feature fusion neural network, the group feature fusion neural network and the global fusion neural network based on the back propagation of the second loss function, iteratively execute the steps of adjusting the neural network based on the back propagation of the first loss function and / or the back propagation of the second loss function until the split neural network converges and the iteration is stopped.

[0024] In an optional embodiment, the application provides a dynamic construction method of a split learning neural network, the server end pre-designs a calculation time threshold, if the time spent by the sub neural network corresponding to the client and the client feature fusion neural network in converging exceeds the calculation time threshold, the server end deletes the neurons in the input layer of the group feature fusion neural network corresponding to the client, and sends the related indication information of deleting the neurons to the client;

[0025] If the deleted client re-sends the request or a new client sends a request, the request carries the state marker information of the client applying to join, the server re-adds the neurons in the input layer of the group feature fusion neural network for the client, when the server end dynamically generates the client feature fusion neural network and the sub neural network for the corresponding client, at least one of the client feature fusion neural network and the sub neural network is compressed, and the existing sub neural network of the corresponding client is updated with the compressed sub neural network, the client feature fusion neural network receives the features sent by the newly added sub network, the newly added client feature fusion neural network receives the features sent by the newly added sub network and generates the first intermediate features after fusion processing, and sends the first intermediate features to the corresponding newly added neurons, the newly added neurons can join the training process, or,

[0026] If the deleted client re-sends the request or a new client sends a request, the request carries the state marker information applied by the client to join, the server dynamically generates a compressed client feature fusion neural network and / or a client generates a compressed sub-neural network according to its own metadata for the corresponding client, the client feature fusion neural network receives the features sent by the new sub-network, the new client feature fusion neural network receives the features sent by the new sub-network and generates first intermediate features after fusion processing, and sends the first intermediate features to the corresponding new neurons, and the new neurons can join the training process.

[0027] In an optional embodiment, the application provides a dynamic construction method of a split learning neural network, and part or all of the clients can also run in the server.

[0028] In an optional embodiment, the application provides a dynamic construction method of a split learning neural network, and the original data is used as the input data of the sub-neural network of the client, and the global fusion sub-network of the server outputs the final prediction value. If a client sends a request to join or exit during the prediction process, for the exited client, the server deletes the neurons in the input layer of the group feature fusion neural network corresponding to the exited client, and notifies the exited client of the information of deleting the related neurons. For the joined client, the server re-adds the neurons in the input layer of the group feature fusion neural network for the client, the new neurons copy the parameters of the neighbor neurons in the input layer of the group feature fusion neural network, the neurons are fully connected with the neurons of the next layer network, the server dynamically generates a client feature fusion neural network and a sub-neural network for the corresponding client again, and sends the sub-neural network to the new client, the client feature fusion neural network receives the features sent by the new sub-network, the new client feature fusion neural network receives the features sent by the new sub-network and generates first intermediate features after fusion processing, and sends the first intermediate features to the corresponding new neurons. Only by re-training the sub-neural network of the new client and the corresponding client feature fusion neural network with the prediction data and the label Y can a new split network be obtained.

[0029] In an optional embodiment, the application further provides a collaborative training device based on longitudinal federated learning, the device comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of the embodiments of the application.

[0030] In an optional embodiment, the present application also provides a storage medium, which is a computer readable storage medium, and the computer readable storage medium stores a program, when the program is executed by a multi-core processor, the multi-core processor executes the method according to any one of the embodiments of the present application.

[0031] To at least partially solve one or more of the above problems and other potential problems, example embodiments of the present disclosure propose a scheme of collaborative training based on longitudinal federated learning. In this scheme, 1, the input layer of the client's sub neural network is adaptively generated according to the metadata and other conditions to be input by the client, which can realize the input of any type and quantity of metadata; 2, the client's sub neural network is adaptively generated according to the metadata and other conditions to be input by the client, so that the time-consuming of the client's sub neural network is dynamically adjustable; 3, by controlling and generating the output layer of the sub neural network in advance, the alignment of data is realized through the neural network, without the need for special alignment of the original data or input data; 4, through the group neural network, the dynamic joining and exiting of the client is realized, making the split neural network more flexible and more in line with the actual situation; 5, through the reuse of prediction data and label Y by the client neural network and global fusion neural network, the split neural network is realized to converge quickly; 6, the calculation time control of the sub neural network, through the comparison of the acceptable calculation time threshold and the calculation time of the sub neural network, the calculation time of the sub neural network is dynamically adjusted and controlled, making the whole network more flexible and practical.

[0032] Other advantages of the present application will be described in more detail in conjunction with the following description and drawings.

[0033] It should be understood that the above description is only a summary of the technical solutions of the present application, so as to enable a clearer understanding of the technical means of the present application, so that it can be implemented according to the content of the specification. In order to make the above and other purposes, characteristics and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are illustrated. BRIEF DESCRIPTION OF DRAWINGS

[0034] The advantages and benefits described herein, as well as other advantages and benefits, will be apparent to those of ordinary skill in the art upon reading the following detailed description of exemplary embodiments. The drawings are for purposes of illustrating exemplary embodiments only and are not to be construed as limiting the present application. Moreover, in the accompanying drawings, like reference numerals designate like parts throughout the various figures. In the drawings:

[0035] Figure 1 The hardware architecture according to an embodiment of the present application is schematically shown in the schematic diagram;

[0036] Figure 2 The method and device flow according to an embodiment of the present application based on longitudinal federated learning collaborative training are schematically shown in the schematic diagram;

[0037] Figure 3 A split network model architecture diagram based on longitudinal federated learning according to yet another embodiment of the present application;

[0038] Figure 4 A supply chain automobile demand prediction method flowchart based on longitudinal federated learning according to yet another embodiment of the present application;

[0039] Figure 5 A split network method of collaborative training longitudinal federated learning for predicting automobile demand according to yet another embodiment of the present application;

[0040] Figure 6 A whole vehicle carbon emission prediction method flowchart based on longitudinal federated learning of collaborative training according to yet another embodiment of the present application;

[0041] Figure 7 A split network method of collaborative training longitudinal federated learning for predicting carbon emissions according to yet another embodiment of the present application.

[0042] In the drawings, identical or corresponding reference signs indicate identical or corresponding parts. DETAILED DESCRIPTION

[0043] Exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood, and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0044] In the description of embodiments of the present application, it should be understood that terms such as "include" or "have" are intended to indicate that there is existence of features, numbers, steps, actions, parts, or combinations thereof disclosed in the specification, and do not exclude the existence or possibility of additional one or more other features, numbers, steps, actions, parts, or combinations thereof.

[0045] Unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" herein is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist together, and B exists alone.

[0046] The terms "first", "second", etc. are only used for descriptive purposes and should not be construed as implying or suggesting relative importance or an indicated number of the technical features. Thus, the features defined with "first", "second", etc. can explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0047] All the codes in the present application are exemplary, and those skilled in the art can think of various modifications according to the used programming language, specific needs and personal habits, etc. without departing from the idea of the present application.

[0048] In order to clearly describe the embodiments of the present application, some concepts that may appear in subsequent embodiments will be introduced first.

[0049] At present, the research on horizontal federated learning is relatively mature, but the research on vertical federated learning is still less, especially the research on vertical federated learning combined with split learning is less. Split learning (Split Neural Networks, SplitNN) is a distributed and private deep learning technology that can train deep neural networks on multiple data sources without directly sharing original labeled data. In the existing split network, after the input data, the input data of the client enters the sub-network of the client, the split neural network is forward propagated for calculation, the sub-network outputs intermediate values and sends them to the server side, the server side synchronizes the intermediate value features of all clients, after synchronization, the server side network is input for calculation, and finally the loss is calculated according to the label. Loss is back-propagated in the split neural network, and is iterated constantly to realize the convergence of the entire split neural network.

[0050] However, the traditional scheme has the following problems:

[0051] In the prior art, the network is a whole, and the front end of the network can only wait while the back end of the network is calculating. Different parts of a deep network cannot be run asynchronously.

[0052] Due to reasons such as confidentiality obligations, information security, and commercial competition, upstream and downstream enterprises in the supply chain are often unwilling to share their own data, resulting in low data sharing degree among upstream, midstream and downstream enterprises, and a large amount of data cannot be fully utilized, resulting in a data island problem. Therefore, how to form a collaborative training and analysis method for data of each enterprise is an urgent problem for enterprises in the value chain.

[0053] Because the computing capabilities of different clients are different, when a faster client transmits data to the server, the server needs to wait for the rest of the clients to transmit data to the server, and after the server waits for data synchronization, the server side can continue to calculate, and asynchronous calculation cannot be realized. Once the number of clients is large, or part of the clients is very slow, it will seriously affect the entire network training and prediction time.

[0054] In the prior art, the network is a whole, and when the front part of the network is calculating, the back end of the network can only wait, and different parts of a deeper network cannot be run asynchronously.

[0055] Each client's sub-network is exactly the same, often requiring input data alignment, without differentiated adjustment according to the input data and calculation time of the client, resulting in the need for input data alignment of different clients, which will cause additional calculation and time consumption. If the data is not aligned or alignment error occurs, calculation cannot be performed, and the entire network is not flexible and cannot adapt to various real situations.

[0056] Once the network of the prior art is built, its network size is fixed and cannot be changed. For some occasions where the network size is not suitable, a large amount of calculation time is required, and the network size cannot be dynamically adjusted according to actual conditions, and the time required for each client to calculate cannot be controlled according to actual needs.

[0057] In practice, due to various reasons such as upstream and downstream cooperation period, business interest changes, client crashes, etc., clients will dynamically exit or enter, but the existing network does not allow clients to dynamically exit or join. On the one hand, once one or more clients exit, the server side network cannot synchronize to the related data, causing synchronization problems. On the other hand, if a client wants to join, but the server and client network in the prior art cannot be expanded, and cannot be compatible with the newly joined client.

[0058] In the prior art, the neural network is split at the end of the network, and the loss function is calculated using labels, and then back propagation is performed. The partial derivative of the loss function with respect to the neuron parameter is calculated to obtain error information, and the error information is transmitted from the server side to all clients. Each client receives the error information and further performs back propagation internally. All clients need to complete back propagation before the next forward propagation can be performed, resulting in a long message synchronization time and a large number of node transmissions, which consumes a lot of time, and the split neural network converges slowly.

[0059] To at least partially address the above-mentioned problems and one or more of other potential problems, example embodiments of this disclosure propose a scheme of collaborative training based on longitudinal federated learning. In this scheme, 1, through the metadata notification server, the server adaptively generates the sub-neural network of the client according to the situation of the original data to be input by the client, and the input layer thereof can realize the input of original data of any name, type, latitude or quantity; 2, different sizes of personalized sub-neural networks are adaptively generated for different clients according to the situation of the clients, so that the time consumption of the client sub-neural network is dynamically controllable; 3, data alignment is realized by controlling the output layer of the sub-neural network in advance, and there is no need to align the data before inputting the original data; 4, through the group neural network, the dynamic joining and exiting of the client is realized, so that the split neural network is more flexible, the system robustness is stronger, the use of the network is more flexible and convenient, and it is more in line with the actual situation; 5, different parts of the network are fully utilized to realize asynchronous operation of the network and accelerate the convergence of the network; 6, by reusing the prediction data and the label Y of the client neural network and the global fusion neural network, the split neural network is realized to quickly converge.

[0060] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0061] Firstly, refer to Figure 1 which schematically shows a schematic diagram of a system architecture diagram in which example implementations according to the present disclosure can be used.

[0062] Figure 1 A schematic diagram of an example of a computing device according to an embodiment of the present disclosure is shown. It should be noted that Figure 1 That is, the present application provides a hardware running environment structure diagram of a dynamic construction method of a split learning neural network in an embodiment of the present application. The client or server device in the embodiment of the present application can be a mobile phone, a tablet computer, a PC, an embedded device, a server, a portable computer, etc.

[0063] As Figure 1As shown, the client can include a communication module, a sub neural network training and / or generation module, a data processing module, a database, a memory, a processor such as a CPU, a network interface, a user interface, and the like, the communication module communicates with the communication module of the server through a communication bus or a wireless transmission module, the sub neural network training and / or generation module is used to generate the sub neural network of the client and / or run the sub neural network. The server can include a communication module, a split neural network training and / or generation module, a parameter management module, a database, a memory, a processor such as a CPU, a network interface, a user interface, the communication module communicates with the communication module of the client through a communication bus or a wireless transmission module, the split neural network training and / or generation module is used to generate the split neural network and run the client feature fusion neural network and the feature fusion network, wherein the feature fusion network includes a plurality of group feature fusion sub-networks and a global fusion sub-network. In an embodiment, the server generates the sub neural network, the client feature fusion neural network and the feature fusion network through the split neural network training and / or generation module, and the server sends the sub neural network to the client, the client runs the sub neural network through the sub neural network training and / or generation module, and the server runs the client feature fusion neural network and the feature fusion network through the split neural network training and / or generation module. In another embodiment, the server generates and runs the client feature fusion neural network and the feature fusion network through the split neural network training and / or generation module, and the client generates and runs the sub neural network through the sub neural network training and / or generation module. The communication bus or the wireless transmission module is used to realize the connection communication between the components. The user interface can include a human-computer interaction module, a display, an input unit such as a keyboard, and the optional user interface can further include a standard wired interface, a wireless interface, or a data input / output interface. The network interface can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface). The memory can be a high-speed RAM memory, or a stable memory (non-volatile memory) such as a disk memory. The memory can optionally be a storage device independent of the aforementioned processor.

[0064] Those skilled in the art can understand that Figure 1 The client or server device structure shown in the foregoing embodiments does not constitute a limitation on the client or server device, and can include more or fewer components than shown, or combine certain components, or different arrangement of components.

[0065] Embodiment 1

[0066] In the value chain, a large amount of data training analysis can save production and operation costs for enterprises, but at present, each enterprise in the value chain pays more and more attention to data privacy and security, and is unwilling to share original data with upstream and downstream enterprises, so a large amount of data cannot be fully utilized, and there is a data island problem. Metadata is data that describes information resources or data objects, and its purpose is to identify resources, evaluate resources, track changes in resources during use, manage a large amount of networked data simply and efficiently, and effectively discover, find, integrate, and manage the use of resources. Therefore, metadata is data that can be publicly disclosed. Metadata includes the characteristics, size, dimension, attributes, generation time, type, shape, client identifier, variable name and dimension of the input data. Metadata is data that describes other data, or structured data that provides information about a certain resource, which can be generated by the client or server side according to the input data or original data of each client. The original data is actual data collected at a specific period, including characteristics and samples; the original data can be collected by rules and methods specified by those skilled in the art, and the original data records the original data of the client in the form of a tensor or a table, including characteristics and samples, for example, characteristic 1 is automobile sales, and samples include the automobile sales data of different clients in January, February, March, …; characteristic 2 is automobile inventory, and samples include the automobile inventory data of different clients in January, February, March, …; characteristic 3 is accessory sales, and samples include the accessory sales data of different clients in January, February, March, …; characteristic 4 is accessory inventory, …, and so on. Those skilled in the art should understand that the characteristics and samples referred to in the present application are not limited to the above, the characteristics can be the characteristics of part or all of the data collected or recorded by each client in practice, and the samples can also be part or all of the data collected or recorded by each client in practice. The characteristics, dimensions, types, etc. of the original data of different clients that can input data can be different, and the characteristics, dimensions, types, etc. of the original data of different groups can also be different, and the client can collect the original data in units of months, weeks, or in real time, and those skilled in the art can arbitrarily set according to actual needs.

[0067] Based on the collaborative training method based on longitudinal federated learning proposed in the present application, the client only needs to share metadata, without sharing original data, protecting the data privacy and security of the enterprise, and making it possible for enterprises in the value chain to safely carry out certain cooperation without sharing data.

[0068] This invention provides a dynamic construction method for split learning neural networks. In an optional embodiment of this invention, a collaborative training method based on vertical federated learning is provided. The established split neural network model includes: a sub-neural network located on each client and multiple client feature fusion neural networks and feature fusion networks located on the server. Preferably, it also includes a prediction data reuse label Y. The server divides multiple clients into multiple groups and dynamically generates a global fusion neural network based on the grouping. It receives requests from multiple clients, each request carrying its own metadata and status markers. The metadata indicates the client's input data, and the status markers indicate whether the client is requesting to join or leave the training process. The training process refers to the construction and training of the split learning neural network on both the server and the clients. Based on the metadata in the requests, the server dynamically generates corresponding sub-neural networks and client feature fusion neural networks for each client and sends the sub-neural networks to the corresponding clients. It also dynamically generates corresponding group feature fusion neural networks for each group based on the number of clients in each group and the status markers in the requests. Alternatively, it dynamically generates corresponding client feature fusion neural networks for each client based on the metadata in the requests, and dynamically generates corresponding group feature fusion neural networks for each group based on the number of clients in each group and the status markers in the requests. Each client dynamically generates its own sub-neural network based on its own metadata. This results in the sub-neural networks, client feature fusion neural networks, group feature fusion neural networks, and the global fusion neural network collectively forming a split neural network.

[0069] like Figure 2 As shown, a dynamic construction device or system for splitting learning neural networks consists of a server S and N clients {C1, C2, ..., C...} i ,…,C N The system consists of a server (S) and a client (C). Here, i ∈ [1, N] and i is a positive integer, with each value of i corresponding to a specific client. N is the number of clients. The order of steps indicates the sequence of events. Arrowed lines represent the nodes where steps are executed or the direction of data flow. The same step number indicates actions performed sequentially on different objects within the same step to complete the same task. i The interaction process includes the following:

[0070] Step S1: The server S sends a message to each client C. iThe metadata acquisition request is sent to acquire metadata related to the task. Preferably, the task is that the automobile production main factory predicts the number of automobiles that should be produced next month, or that the parts manufacturers of the joint vehicle system accurately predict the corresponding carbon emissions according to the parts data.

[0071] Step S2: Each client C i Metadata is sent to the server side. In another optional embodiment, in step S2, each client C i The server side receives the request sent by the client, and the request carries the metadata of each client, state marker information, and the state marker information is used to indicate that the client decides to apply for joining or quitting the task; or, in another optional embodiment of the application, the request sent by the client carries the state marker information of each client, and the state marker information is used to indicate that the client applies for joining or quitting the training process. The metadata is a feature description of each enterprise input data, and the metadata is a data structure in the form of an array or a tensor, which records the features, size, latitude, attribute, generation time, type, shape and other information of each input data or original data of the client. The server side has data labels Y, and the label Y is the historical production record of the main factory or the original data obtained by manually labeling the historical production record. The original data includes inventory, sales, customer data, parts data and other samples, and in a preferred embodiment, the original data can be all data related to automobile production in practice. The data can be aggregated according to the time period t by the server side or the main factory and the client to obtain the input data set. Since the generation time of the original data is uncertain, data within a certain time period can be collected. Preferably, the time period t is, for example, 7 days, indicating that the aggregation is performed in weeks, or the aggregation can be performed every day, and the specific size of t can be set by the main factory or the server or the user or the client according to actual needs. The intermediate data obtained by processing the original data is used to construct the input data set, and the input data set can be stored in the local database of the client. The input data can be composed of the features and samples of the data, represented by X i The tensor can be a vector, a matrix or a high-dimensional data structure.

[0072] Step S3: the server groups the clients and generates the split neural network. The server generates a sub neural network for each client according to the metadata of the client, preferably, the sub neural network is implemented by using a multi-layer perceptron or a recurrent neural network or a Transformer, and sends the sub neural network to the corresponding client; meanwhile, a plurality of client feature fusion neural networks and a feature fusion network located at the server are generated, the feature fusion network includes a plurality of group feature fusion sub-networks and a global fusion sub-network, so that the sub neural network, the client feature fusion neural network and the feature fusion network jointly constitute the split neural network. The input layer of the sub neural network of each client is generated according to the metadata of each client, the number of neurons is equal to the dimension of the input data, and the dimension is the number of features. The input layer of the sub neural network corresponding to each client is different according to the metadata of each client, and any data of the corresponding client can be input. The output layer of all the sub neural networks has the same number of neurons, preferably, the output layer of all the sub neural networks has R neurons, R is a positive integer, and a fixed value can be set by a person skilled in the art according to the training condition. The sub neural network extracts features from the input data and sends the extracted features to the client feature fusion neural network. The client feature fusion neural network extracts features by using a multi-layer perceptron or a Transformer, the second last layer or the third layer obtains a first intermediate feature, and the last layer obtains a predicted value of a label. A first loss function is calculated according to the predicted value of the label and a multiplexing label Y, and the extracted first intermediate feature is output to the group feature fusion neural network. The group feature fusion sub-network is dynamically generated according to the grouping of the clients and the number of clients in the group. The server can increase or delete the neurons corresponding to the changed clients in the input layer of the group feature fusion sub-network according to the change of the clients in the group. The added neurons in the input layer are fully connected with the neurons in the adjacent layer (i.e. the second layer of the group feature fusion sub-network), and the remaining part of the group feature fusion sub-network remains unchanged. The input layer of the group feature fusion sub-network fuses a plurality of first intermediate features, extracts features by using a multi-layer perceptron or a Transformer, and outputs a second intermediate feature through the output layer. The input layer of the global fusion sub-network is dynamically generated according to the grouping. The server can increase or delete the neurons corresponding to the changed groups in the input layer of the global fusion sub-network according to the change of the groups. The input layer of the global feature fusion sub-network fuses a plurality of second intermediate features by using a multi-layer perceptron or a Transformer, and extracts high-order interactions between the second intermediate features between the groups. In an optional embodiment of the present application, all the sub neural networks, client feature fusion neural networks, group feature fusion sub-networks and global fusion sub-networks are implemented by using a multi-layer perceptron.In an optional embodiment of the present application, all sub neural networks, client feature fusion neural networks, and group feature fusion sub networks are implemented using multi-layer perceptron, and all global fusion sub networks use Transformer. In an optional embodiment of the present application, the global fusion sub network can be built in the same mode as the group feature fusion sub network. The global fusion sub network is dynamically generated according to the grouping of the clients, and the server can add or delete neurons corresponding to the changed groups in the input layer of the global fusion sub network according to the change in the number of groups. The added neurons in the input layer are fully connected with the neurons in the next adjacent layer (i.e. the second layer of the global fusion sub network), and the rest of the global fusion sub network remains unchanged. The input layer of the global fusion sub network uses a convolution block to fuse multiple second intermediate features, uses one or two residual modules for feature extraction, and outputs a prediction value and a second loss function through the output layer. In an optional embodiment of the present application, each client generates a corresponding sub neural network according to its own metadata, without having to generate a sub neural network by the server. In an optional embodiment, there is a client running in the server, and the server establishes a sub model M0 for the client running in the server, and the sub model M0 is responsible for processing the local feature data of the server. At the same time, a sub model M is established for each client respectively. i (i = 1, 2, …, N); the server sends the client sub model M i to the corresponding client respectively. In the present application, for the sake of brevity, the client running on the server and the client not running on the server are handled together, but those skilled in the art should understand that the client in the present application can be partially or entirely running in the server. In an optional embodiment of the present application, the N clients are divided into M groups, and M group feature fusion sub networks, one global fusion sub network, and one prediction data multiplexing label Y are generated according to the grouping. The M group feature fusion sub networks correspond to the M groups one by one, and each group contains a plurality of clients and a client feature fusion neural network corresponding to the clients one by one. Preferably, the same type of enterprise clients are divided into a group and k is a positive integer, the value of k corresponds to different groups one by one, and M is the number of groups of the grouping. Preferably, the grouping condition of the clients can be grouped by identity, for example, first-level suppliers are grouped as a group, and second-level suppliers are grouped as another group, and third-level suppliers, fourth-level suppliers, and the like are grouped in the same way. Preferably, the grouping condition can be grouped by the region to which the client belongs, for example, grouped by the south, north, west, and east regions; or grouped by different cities where the clients are located, and the like. Those skilled in the art can formulate the grouping condition of the clients according to the needs, and the present application does not limit this.

[0073] In an optional embodiment of the present application, the present application provides a dynamic construction method for splitting a learning neural network, which comprises model training: first training a sub neural network model of a client and a client feature fusion neural network of a server, and then training a feature fusion network model of the server. Each client first trains its own sub neural network model, and for each group, a plurality of client feature fusion neural networks and a feature fusion module are trained on the server, wherein the feature fusion network comprises a plurality of group feature fusion sub networks and a global fusion sub network; in the case of grouping a plurality of clients, each group of clients corresponds to a group feature fusion sub network; the group feature fusion sub network fuses the first intermediate features output by the client feature fusion neural networks in the group. The global fusion sub network is trained on the server, and the global fusion sub network fuses the second intermediate features output by the group feature fusion sub networks of each group.

[0074] Step S4: Each client first trains a sub model M i , which is a model composed of a sub neural network running in the sub neural network training and / or generation module of the client. Each client takes the original data X i stored in its own database as input data of the sub model, and performs forward propagation along the sub neural network to obtain a feature vector and sends it to the server. The client feature fusion neural network of the server calculates the predicted value Y of the prediction data multiplexing label using the feature vector sent by the client. i (r) The server calculates the first loss function L based on the first loss function L iThe encrypted data is then sent to the client. In another optional embodiment, the last layer of the client sub neural network receives the loss After that, the gradient is calculated and backpropagation is performed, and the process is iterated according to step S4, and a threshold value of the number of iterations is set, preferably, the maximum value of r is 20 rounds. The sub model M i After the training is completed, the client C i uses M i The feature vectors of all samples (or input data) are calculated and sent to the server. The client feature fusion neural network of the server calculates the predicted value according to the feature vector. In an optional embodiment of the present application, the server receives the output value of the sub neural network and inputs the output value into the client feature fusion neural network, and then determines the first intermediate feature fusion function value (also called the predicted value of the label in the present application) and the first loss function through the client feature fusion neural network and the multiplexed label Y. The server and the client adjust at least one of each sub neural network and the client feature fusion neural network based on the backpropagation of the first loss function, and the process is repeated until the sub neural network and the client feature fusion neural network converge, and the step is stopped. The client feature fusion neural network outputs the first intermediate feature and sends it to the corresponding group feature fusion sub network; the first intermediate feature can be output by the network layer of the client feature fusion neural network that extracts the feature, such as the penultimate layer of neural network or the antepenultimate layer of neural network, and those skilled in the art can select the network layer that can extract the specific feature according to the actual needs, which is not limited in the present application. Figure 3 As shown in the figure, for example, the penultimate layer of neural network or the antepenultimate layer of neural network, those skilled in the art can select the network layer that can extract the specific feature according to the actual needs, which is not limited in the present application.

[0075] By multiplexing the label in the client feature fusion neural network, the asynchronous processing of the split neural network is realized asynchronously through the label. On the one hand, the client sub neural network and the client feature fusion neural network do not affect the calculation of the feature fusion network (including multiple group feature fusion sub networks and one global fusion sub network) when iteratively calculating, and different sub networks of the entire network can not affect each other when independently performing iteration, realizing the asynchronous processing of the front and back of the network. On the other hand, the second loss function does not have to be transmitted to each client, reducing the calculation on the node and saving the operation time, and accelerating the network convergence speed.

[0076] In an optional embodiment, the present application provides a dynamic construction method of split learning neural network. For each client, the sub neural network runs in the sub neural network training and / or generation module, and in each round of iterative training, the sub neural network training and / or generation module obtains the original data or input data X i of the current iteration from the database of the client to which it belongs. iThe intermediate data obtained by processing the original data can also be used. One round of inference calculation and training is completed on each sub neural network, and then the output of the sub neural network is transmitted to the client feature fusion neural network on the server side through the communication module. The server side calculates the first intermediate feature fusion function value F1(X i ) and the first loss function L1(X i ) corresponding to each client through the client feature fusion neural network and the shared prediction data label Y, and performs back propagation to update the parameters P j of the sub network model, where j is a positive integer representing the number corresponding to each neuron in each sub network. The server side and the client adjust at least one of each sub neural network and the client feature fusion neural network based on the back propagation of the first loss function, and continuously iterate until the sub neural network and the client feature fusion neural network converge or reach the maximum number of iterations or the iteration time exceeds the preset calculation time threshold, and then stop iteration. The client feature fusion neural network outputs the first intermediate feature and sends it to the corresponding group feature fusion sub network. The first intermediate feature is output by the network layer of the client feature fusion neural network that extracts data features, and preferably, the network layer of the client feature fusion neural network that extracts data features is the second last layer or the third last layer.

[0077] Step S5: The server side generates a second intermediate feature by fusing at least one first intermediate feature in the group through the group feature fusion neural network, and establishes a group feature fusion sub network for each group on the server side for fusing the outputs of the client feature fusion neural networks corresponding to each client in the group. When all client feature fusion neural networks in a group are trained, the group feature fusion sub network of each group is trained to fuse the first intermediate features output by the client feature fusion neural networks corresponding to the clients in the group and obtain the second intermediate feature, and send the second intermediate feature to the global fusion sub network. In an optional embodiment of the present application, the server side adds or deletes neurons in the input layer of the group feature fusion sub network for dynamically added or exited clients in the group according to the state marker information carried in the request sent by the client. The neurons are used to receive the first intermediate features output by the client feature fusion neural networks. The group feature fusion sub network receives the output values of the client sub neural networks corresponding to all clients in the group and fuses them to obtain the second intermediate feature, and inputs the second intermediate feature into the global fusion sub network.

[0078] In an optional embodiment of the present application, the present application provides a dynamic construction method for splitting a learning neural network, wherein the step S5 further comprises: the server side pre-designs a calculation time threshold, and judges whether the time spent by the client side corresponding to the sub neural network and the client side feature fusion neural network for convergence exceeds the calculation time threshold, and if so, the server side deletes the neurons corresponding to the client side in the input layer of the group feature fusion sub network, and sends relevant indication information of deleting the neurons to the client side, and if the deleted client side re-sends the request, the server re-adds the neurons in the input layer of the group feature fusion sub network for the client side, and simultaneously, the server side dynamically generates the client side feature fusion neural network and the sub neural network with smaller neurons for the corresponding client side, or the server side dynamically generates the client side feature fusion neural network with less time for the corresponding client side, and the client side generates the sub neural network with less time according to the metadata information of the client side. If the deleted client side re-sends the request or a new client side joins and sends the request, the request carries the metadata and state marker information of the client side, the server re-adds the neurons in the input layer of the group feature fusion neural network for the client side according to the state marker information in the request, and the server side dynamically generates the client side feature fusion neural network and the sub neural network for the corresponding client side, and at least one of the client side feature fusion neural network and the sub neural network is compressed to reduce the time spent by the client side in the training process, the client side feature fusion neural network receives the features sent by the added sub network, the added client side feature fusion neural network receives the features sent by the added sub network and generates first intermediate features after fusion processing, and sends the first intermediate features to the corresponding added neurons, and the added neurons can join the training process; or, if the deleted client side re-sends the request or a new client side joins and sends the request, the request carries the state marker information of the client side, the server re-adds the neurons in the input layer of the group feature fusion neural network for the client side according to the state marker information in the request, the server side dynamically generates the compressed client side feature fusion neural network for the corresponding client side, and / or the client side generates the compressed sub neural network according to the metadata of the client side, the client side feature fusion neural network receives the features sent by the added sub network, the added client side feature fusion neural network receives the features sent by the added sub network and generates first intermediate features after fusion processing, and sends the first intermediate features to the corresponding added neurons, and the added neurons can join the training process.

[0079] Through the above steps, if there are problems such as expiration of the upstream and downstream cooperation period, change of commercial interests, client crash, etc., they can be well adapted, and the dynamic exit or entry of the client is realized, and the client and the network can be dynamically expanded, so that the network is more compatible and robust. Some clients that require a lot of calculation time are deleted, so that clients within the calculation time threshold do not have to wait for clients that take a long time to calculate, saving calculation time, and for clients whose data are important, they can join the task in time by re-sending the request. In addition, the network size can also be dynamically adjusted according to the situation, such as reducing the number of layers of the neural network or reducing the number of neurons in a certain layer, etc., so as to control the time required for each client to operate according to actual needs.

[0080] Step 6: The server determines the predicted value and the second loss function through the global fusion sub-network and the prediction data multiplexing label Y, and adjusts at least one of each of the sub-neural networks, the client feature fusion neural network, the group feature fusion sub-network and the global fusion sub-network based on the back propagation of the second loss function. Steps 4 to 6 are repeatedly executed until the split neural network converges or the maximum number of iterations is reached or the time spent exceeds the maximum time threshold, after which steps four to six are stopped. The maximum number of iterations or the maximum time threshold can be preset in advance by a person skilled in the art according to actual needs.

[0081] Step 7: The final predicted value is output by the global fusion sub-network of the server through the input of the original data or the intermediate data obtained by processing the original data by the sub-neural network of the client. Preferably, for example, the final predicted value represents the number of cars that should be produced by the host factory next month. In an optional embodiment, during the prediction process, if a client sends a request to join or exit, for the exiting client, the server deletes the neurons in the input layer of the group feature fusion neural network corresponding to the exiting client, and notifies the exiting client of the information about the deletion of the relevant neurons. For the joining client, the server re-adds the neurons in the input layer of the group feature fusion neural network for the client, the newly added neurons copy the parameters of the neighboring neurons in the input layer of the group feature fusion neural network, and the neurons are fully connected with the neurons of the next layer network. The server dynamically generates the client feature fusion neural network and the sub-neural network corresponding to the client, and sends the sub-neural network to the newly added client. The client feature fusion neural network receives the features sent by the newly added sub-network, and the newly added client feature fusion neural network receives the features sent by the newly added sub-network and generates the first intermediate features after fusion processing. The first intermediate features are sent to the corresponding newly added neurons. Only the sub-neural network and the corresponding client feature fusion neural network of the newly added client need to be re-trained using the prediction data multiplexing label Y to obtain a new split network.

[0082] The upstream and downstream enterprises in the supply chain do not need to share their original data, and the influence of the original data of the upstream and downstream enterprises is considered to form data collaborative training analysis of each enterprise, so that the final value prediction is more accurate, and the information island dilemma is broken.

[0083] In an optional embodiment of the present application, the present application provides a dynamic construction method of a split learning neural network, in step three of the method, the input layer of the sub neural network is dynamically generated according to the metadata to be input by the corresponding client, and all output layers of the sub neural network have the same number of neurons, which is preset by the server side or negotiated among the multiple clients.

[0084] In an optional embodiment of the present application, the present application provides a dynamic construction method of a split learning neural network, in the method, part or all of the clients can run in the server.

[0085] In an optional embodiment of the present application, the present application provides a dynamic construction method of a split learning neural network, in the method, the metadata records the data input by the client in the form of a tensor, including the characteristics, size, latitude, attribute, generation time, type, shape of the input data, and the metadata indication information includes the client flag, the metadata type and dimension to be input.

[0086] In an optional embodiment of the present application, the present application further provides a dynamic construction device of a split learning neural network, the device comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of the embodiments of the present application.

[0087] In an optional embodiment of the present application, the present application further provides a computer readable storage medium, the computer readable storage medium stores a program, when the program is executed by a multi-core processor, the multi-core processor executes the method according to any one of the embodiments of the present application.

[0088] Embodiment 2

[0089] According to the embodiments of the present application, a dynamic construction method of a split learning neural network and a supply chain demand prediction method based on collaborative training of vertical federated learning are provided, and the vertical federated learning idea refers to federated learning divided by features, which is suitable for a federated learning scenario composed of participants with the same sample space and different feature spaces on a data set.

[0090] Figure 4This is a supply chain demand forecasting method based on vertical federated learning according to an embodiment of the present invention, such as... Figure 4 As shown, the method includes the following steps:

[0091] S201, Obtain characteristic data of upstream and downstream enterprises in the supply chain. The characteristic data is raw data, which is input through the client's sub-neural network.

[0092] Based on the concept of vertical federated learning, feature data from upstream, midstream, and downstream enterprises in the supply chain are combined to train a federated model. Upstream suppliers provide raw data such as orders, inventory, production status, and shipment quantities; midstream distributors provide raw data such as orders and inventory; and downstream retailers provide raw data such as sales and inventory. The raw data is statistically analyzed at specific time intervals, for example, sampling values ​​within a one-week time span.

[0093] The manufacturing enterprise is treated as a client running on a federated learning server S, while simultaneously maintaining a client model C. s Preferably, the server S can be owned by the manufacturing enterprise, or more preferably, the server S can be a server rented by the manufacturing enterprise from a third-party company. Distributors, retailers, and other suppliers at various levels are used as clients C in the federated learning process. i (i = 1, 2, ..., n), where n represents the number of clients. Preferably, C s It can be C i A client. Preferably, C s It can be independent of C i One client. Local data from each client is stored in the local database through the data processing module.

[0094] S202, Deconstructing the establishment and distribution of neural network models;

[0095] The server-side S is for each enterprise client C. i Establish a sub-model M i And through the communication module, the sub-neural network M i Send to the corresponding client. In an optional embodiment, the sub-neural networks M of all clients... i The sub-neural networks are built by the client itself and can send the output values ​​of the sub-neural networks to the server. In an optional embodiment, some enterprise clients' sub-neural networks are built by the enterprise clients themselves and can send the network's output values ​​to the server, while the sub-neural networks of other enterprise clients are built by the server S for each enterprise client C. i Establish and transmit the sub-neural network M through the communication module i Send to the corresponding client.

[0096] Then, clients are grouped into different types of groups according to different supply levels (e.g., retailer group G). d Distributor Group G a Manufacturing Plant Group G m The server-side S represents G. d G a and G m A group feature fusion subnetwork M is established. G，d M G，a M G，m The outputs of the client feature fusion neural networks corresponding to the sub-neural networks of each client in the group are used to fuse the outputs of the client feature fusion neural networks. Each group feature fusion sub-network, as well as the clients in the group and the client feature fusion neural networks corresponding to the clients in the group, belong to the same group. The group contains the sub-neural networks located on the client side, as well as multiple client feature fusion neural networks and multiple group feature fusion sub-networks located on the server side. All the neural networks in a group constitute a local fusion model.

[0097] Then, establish a globally fused subnetwork M on the manufacturing plant server side. global It is used to fuse the outputs of each local fusion model. All sub-neural networks, client feature fusion neural networks, group feature fusion sub-networks and global fusion sub-networks together form the global model M.

[0098] S203, Model Training;

[0099] Based on the demand forecasting task, retailers, distributors, and manufacturers, among other clients, combine dynamic encryption algorithms and utilize the collaborative training method of vertical federated learning disclosed in this invention to collaboratively train the split network model established in step S202. For example... Figure 5 As shown, the training process can be divided into the following four steps:

[0100] S203_1: Retailers, distributors, and manufacturers' clients use the raw data obtained in step S201 to optimize the sub-neural network M. i Training is performed. Input the original data X. i Along M i Forward propagation yields sub-feature vectors And The data is sent to the server. The sub-feature vector is the output feature of the client's sub-neural network. The server utilizes... Calculate the predicted value Y of product demand. i After r iterations, the sub-feature vectors can be obtained. and predicted values Preferably, such as Figure 3 As shown, the sub-feature vector From sub-neural network M iThe second last layer or the third last layer is obtained.

[0101] S203_2: Obtain the predicted value Y of each client's demand in each round of training i Afterwards, the server calculates a first loss L1 by using the client feature fusion neural network and the prediction data multiplexing label Y i Then, the homomorphic encryption algorithm is used to encrypt L1 i After encryption, L1 is sent to the client. After receiving L1 i The client calculates the gradient and performs back propagation to adjust the client's sub neural network. In another embodiment, based on L1 i The gradient is sequentially back propagated to the corresponding client feature fusion neural network and the client's sub neural network to adjust at least one of the client feature fusion neural network and the sub neural network.

[0102] The server side pre-designs a calculation time threshold. If the time spent by the client's corresponding sub neural network and the client feature fusion neural network to converge exceeds the calculation time threshold, the server side deletes the neurons in the input layer of the group feature fusion neural network corresponding to the client, and sends relevant indication information of deleting the neurons to the client;

[0103] If the deleted client re-sends the request or a new client joins and sends a request, the request carries state marker information of the client applying to join. The server re-adds the neurons in the input layer of the group feature fusion neural network for the client. When the server side dynamically generates the client feature fusion neural network and the sub neural network for the corresponding client, at least one of the client feature fusion neural network and the sub neural network is compressed, and the compressed sub neural network is used to update the existing sub neural network of the corresponding client. The client feature fusion neural network receives the features sent by the added sub network. The added client feature fusion neural network receives the features sent by the added sub network and generates first intermediate features after fusion processing. The first intermediate features are sent to the corresponding added neurons. The added neurons can join the training process, or

[0104] If the deleted client re-sends the request or a new client joins and sends a request, the request carries state marker information of the client applying to join. The server side dynamically generates a compressed client feature fusion neural network for the corresponding client and / or the client generates a compressed sub neural network according to its own metadata. The client feature fusion neural network receives the features sent by the added sub network. The added client feature fusion neural network receives the features sent by the added sub network and generates first intermediate features after fusion processing. The first intermediate features are sent to the corresponding added neurons. The added neurons can join the training process.

[0105] S203_3: When the client sub-neural network and the client feature fusion neural network in one group are trained, the client feature fusion neural network outputs the first intermediate feature and sends it to the group feature fusion sub-network, then the corresponding local fusion model of the group is trained, and the first intermediate features corresponding to the clients in the group are fused to determine the second intermediate feature

[0106] S203_4: All local fusion models and global fusion sub-networks jointly constitute a global model M, and the second intermediate feature is input into the global fusion sub-network to obtain a global demand prediction value Z2, wherein 1 is the iteration number of the second intermediate feature, and the value of k corresponds to different groups one by one, that is, k corresponds to the number of groups in the group, and the server end calculates the second loss L2 using the prediction data multiplexing label Y i , based on the second loss function, at least one of the sub-neural network, the client feature fusion neural network and the feature fusion neural network is adjusted, and the step of continuously cyclically adjusting the global model is repeated to obtain the second intermediate feature wherein q is the iteration number of the second intermediate feature, and the value of k corresponds to different groups one by one, that is, k corresponds to the number of groups in the group, and the sub-neural network, the client feature fusion neural network and the feature fusion neural network are adjusted according to the second loss function. Preferably, the gradient is calculated by the second loss function, and the multiple group feature fusion sub-networks and the global fusion sub-network are adjusted by back propagation.

[0107] S204, demand prediction is performed according to the global model M.

[0108] The sub-neural network, the client feature fusion neural network, the group feature fusion sub-network and the global fusion sub-network jointly constitute the global model M. The clients of retailers, distributors and manufacturers input local original data into the global model M and perform a forward propagation to obtain a demand prediction value

[0109] In an optional embodiment, in the process of prediction, if a client sends a request to join or exit, for the exiting client, the server deletes the neurons corresponding to the exiting client in the input layer of the group feature fusion neural network of the group, and notifies the exiting client of the information of the deleted neurons. For the joining client, the server re-adds the neurons in the input layer of the group feature fusion neural network of the group, the added neurons copy the parameters of the neighbor neurons in the input layer of the group feature fusion neural network, the neurons are fully connected with the neurons of the next layer network, the server dynamically generates the client feature fusion neural network and the sub neural network corresponding to the client, and sends the sub neural network to the added client, the client feature fusion neural network receives the features sent by the added sub network, the added client feature fusion neural network receives the features sent by the added sub network and generates the first intermediate features after fusion processing, and sends the first intermediate features to the corresponding added neurons. Only by retraining the sub neural network of the added client and the corresponding client feature fusion neural network with the prediction data multiplexing label Y can the new split network be obtained.

[0110] Through the above steps, a longitudinal federated learning model is established based on multi-party private data to perform demand prediction. The channel for collaborative planning, prediction and replenishment from an upstream manufacturer to a downstream retailer is opened, and terminal sales data is timely and accurately transmitted to the manufacturer, so that accurate demand prediction is realized, and problems such as large inventory backlog, poor capital turnover, extended delivery period and logistics in the development process of the supply chain are solved, and the competitiveness of enterprises is improved.

[0111] Embodiment 3

[0112] A car is composed of thousands of parts, and different parts are produced by different enterprises. The part manufacturers have complete part attribute data, but the prediction of carbon emissions based on the data fails to consider the carbon emissions generated by the assembly of parts, and the result is often not accurate enough. According to the idea of longitudinal federated learning, the feature data of the part manufacturers involved in the whole vehicle are combined to train a federated model.

[0113] According to the embodiments of the present application, a dynamic construction method of a split learning neural network and a whole vehicle carbon emission prediction method based on collaborative training of longitudinal federated learning are provided. As shown in Figure 6 The method comprises the following steps:

[0114] S301, obtaining part data and carbon emission data thereof, the part data and the carbon emission data thereof constituting original data;

[0115] The whole vehicle enterprise is taken as a server S of federated learning, and a client model C is maintained at the same time sVarious component manufacturers are taken as clients C of federated learning i (i = 1, 2, …, n), where n represents the number of clients, preferably, C s may be a client C i , where s ∈ [1, n], n is a positive integer greater than 1. Preferably, C s may be a client C i . The raw data of each client is stored in the database of the client through the data processing module.

[0116] S302, establishment and distribution of the split neural network model;

[0117] The server S establishes a sub-model M i for each component client C i , and sends the sub-model M i to the corresponding client through the communication module. In an optional embodiment, the sub-neural network M i of all clients is established by the clients themselves and the output value of the network can be sent to the server. In an optional embodiment, the sub-neural network M i of part of the enterprise clients is established by the enterprise clients themselves and the output value of the network can be sent to the server, and the sub-neural network M i of the other part of the enterprise clients is established by the server S for each enterprise client C i and sent to the corresponding client through the communication module. i

[0118] Then the clients are divided into different types of groups G k (k = 1, 2, …, m), G represents a group, k is the number of the group, and m represents the number of groups. The server S respectively establishes a local fusion model M k for each group G G , k for fusing the outputs of the client feature fusion neural networks corresponding to the sub-neural networks of the clients in the group. Each group feature fusion sub-network, the clients in the group, the sub-neural networks of the clients, and the client feature fusion neural networks corresponding to the clients in the group belong to a group. The group includes the sub-neural networks located in the clients and the multiple client feature fusion neural networks and multiple group feature fusion sub-networks located in the server. All the neural networks in a group constitute a local fusion model.

[0119] Then a global fusion sub-network M global ​It is used to fuse the outputs of each local fusion model. All sub-neural networks, client feature fusion neural networks, group feature fusion sub-networks and global fusion sub-networks together form the global model M.

[0120] S303, Model Training;

[0121] Based on the carbon emission prediction task, each component client uses the collaborative training method of vertical federated learning disclosed in this invention to collaboratively train the vertical federated learning model established in step S302. For example... Figure 7 As shown, the training process can be divided into the following four steps:

[0122] S303_1: Each component client uses the raw data obtained in step S301 to process the sub-neural network M. i Training is performed. Input the original data X. i Along M i Forward propagation is performed to obtain sub-feature vectors And Send to the server. The server uses... Calculate the predicted value of carbon emissions Y i After r iterations, the sub-feature vectors can be obtained. And carbon emission forecasts Preferably, such as Figure 3 As shown, the sub-feature vector From sub-neural network M i Obtained from the second to last or third to last level.

[0123] S303_2: The predicted carbon emissions Y of each component client is obtained in each round of training. i Subsequently, the server calculates the first loss L1 by fusing client features with a neural network and reusing the label Y using historical carbon emission prediction data. i Then, use a homomorphic encryption algorithm on L1 i It is encrypted before being sent to the client. The client receives L. i The gradient is then calculated and backpropagation is performed to adjust the sub-neural network of the client. In another embodiment, based on L1... i The gradient is calculated and propagated backwards sequentially to the corresponding client feature fusion neural network and the client sub-neural network, thereby adjusting the client feature fusion neural network and the sub-neural network.

[0124] S303_3: When at least one of the client sub-neural network and the client feature fusion neural network within a group has been trained, the client feature fusion neural network outputs a first intermediate feature. and sends to the group feature fusion sub-network, then the corresponding local fusion model of the group is trained, and the corresponding first intermediate feature of the client in the group is fused The second intermediate feature is determined

[0125] S303_4: All local fusion models and global fusion sub-networks jointly constitute a global model M, and the second intermediate feature is input into the global fusion sub-network to obtain a global carbon emission prediction value Z2, and the server end calculates a second loss L2 using the prediction data and the label Y i , and at least one of the sub-neural network, the client feature fusion neural network and the feature fusion neural network is adjusted based on the second loss function. The step of adjusting the global model is iteratively repeated to obtain the second intermediate feature , wherein q is the iteration number of the second intermediate feature, the value of k corresponds to different groups one by one, that is, k corresponds to the number of groups of the group, and at least one of the sub-neural network, the client feature fusion neural network and the feature fusion neural network is adjusted based on the second loss function. Preferably, the gradient is calculated by the second loss function, and the multiple group feature fusion sub-networks and the global fusion sub-network are adjusted by back propagation.

[0126] S304, carbon emission prediction is performed according to the global model M.

[0127] The sub-neural network, the client feature fusion neural network, the group feature fusion sub-network and the global fusion sub-network jointly constitute a global model M. Each component client inputs the local component data into the global model M and performs a forward propagation to obtain a carbon emission prediction value In the process of prediction, if a client sends a request to join or quit, for the quitting client, the server deletes the neurons corresponding to the quitting client in the input layer of the group feature fusion neural network, and notifies the quitting client of the information of the deleted neurons. For the joining client, the server re-adds neurons in the input layer of the group feature fusion neural network, the added neurons copy the parameters of the neighbor neurons in the input layer of the group feature fusion neural network, and the neurons are fully connected with the neurons of the next layer network, the server dynamically generates the client feature fusion neural network and the sub neural network corresponding to the client, and sends the sub neural network to the added client, the client feature fusion neural network receives the features sent by the added sub network, the added client feature fusion neural network receives the features sent by the added sub network and generates the first intermediate feature after fusion processing, and sends the first intermediate feature to the corresponding added neurons, only the sub neural network and the corresponding client feature fusion neural network of the added client need to be retrained by using the prediction data multiplexing label Y to obtain the new split network.

[0128] Through the above steps, a longitudinal federated learning model is established based on multi-party private data to predict carbon emissions. Distributed training is performed on the part data of the part manufacturers of each subsystem in the whole vehicle, thereby realizing accurate carbon emission prediction.

[0129] It should be noted that the steps not described in detail in the embodiments can refer to the descriptions of the related steps in the embodiments shown in the Figure 1 The descriptions of the related steps in the embodiments shown in the

[0130] In the description of the present specification, the description of the terms "some possible embodiments", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any suitable manner in any one or more embodiments or examples. In addition, the skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0131] With regard to the method flow diagrams of the embodiments of the present application, certain operations are described as distinct steps within the flow diagrams. Such flow diagrams are illustrative rather than restrictive. Certain steps described herein can be grouped together and performed in a single operation, certain steps can be split into multiple sub-steps, and certain steps can be performed in an order different from that shown herein. The individual steps illustrated in the flow diagrams can be implemented in any manner by any circuit structure and / or tangible mechanism (e.g., by software running on a computer device, hardware (e.g., a processor or a chip-implemented logic function), etc., and / or any combination thereof) in any manner.

[0132] Those skilled in the art will appreciate that embodiments of the present application can be readily used as a method, a system or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) embodying computer program code thereon for use by or in connection with an instruction execution system. These computer program products can provide data

[0133] The present application is described in reference to the flow diagrams and / or block diagrams of the method, apparatus (system) and computer program product according to the embodiments of the present application. It is understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing machine, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more functions specified by one or more blocks Figure 1 one or more functions specified by one or more blocks.

[0134] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 one or more functions specified by one or more blocks Figure 1 one or more functions specified by one or more blocks. In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0135] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology for storing information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. In addition, although the operations of the method of the present application are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in that particular order, or that all of the shown operations must be performed to achieve the desired result. Additionally or alternatively, certain steps can be omitted, a plurality of steps can be combined into one step, and / or one step can be divided into a plurality of steps.

[0136] Although the spirit and principles of the present application have been described with reference to several specific embodiments, it should be understood that the present application is not limited to the disclosed specific embodiments, and the division of aspects does not mean that the features in these aspects cannot be combined to benefit. This division is only for the convenience of expression. The present application is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the appended claims.

Claims

1. A method for dynamically constructing a split learning neural network, characterized in that, a server divides a plurality of clients into a plurality of groups, and dynamically generates a global fusion neural network according to the grouping; the server receives requests sent by the plurality of clients, wherein each request sent by each client carries metadata of the corresponding client and state marker information, the metadata is used to indicate information of input data of the client, and the state marker information is used to indicate that the client applies to join or exit a training process, wherein the training process refers to a process of constructing and training a split learning neural network at the server and the clients; the server dynamically generates a corresponding client feature fusion neural network for each client according to the metadata in the request, and dynamically generates a corresponding group feature fusion neural network for each group according to the number of clients in each group and the state marker information in the request; alternatively, the server dynamically generates a corresponding client feature fusion neural network for each client according to the metadata in the request, and dynamically generates a corresponding group feature fusion neural network for each group according to the number of clients in each group and the state marker information in the request, and each client dynamically generates a corresponding sub neural network according to its own metadata; so that the sub neural networks, the client feature fusion neural networks, the group feature fusion neural networks and the global fusion neural network jointly constitute a split neural network.

2. The method of claim 1, wherein, The dynamic generation of the corresponding sub neural network further comprises: the number of input layer neurons of the sub neural network is dynamically generated according to the metadata of each client, and the number of input layer neurons of the sub neural network is equal to the dimension of the input data of the corresponding client, and the same number of neurons is dynamically generated for the output layer of all the sub neural networks, and the number of output layer neurons is preset by the server in advance or set by negotiation among the plurality of clients. 3.The method of claim 1, characterized in that, The server end receives the output value of the sub neural network and inputs the output value into the client feature fusion neural network, and then multiplexes the label through the client feature fusion neural network and prediction data determining a first intermediate feature fusion function value and a first loss function, adjusting at least one of the respective sub neural networks, the client feature fusion neural network based on the back propagation of the first loss function by the server end and the client, iteratively performing the step of adjusting the neural network based on the back propagation of the first loss function until the sub neural network and / or the client feature fusion neural network converges, and stopping the iteration, the client feature fusion neural network outputs a first intermediate feature and sends it to the corresponding group feature fusion neural network.

4. The method of claim 3, wherein, the dynamic generation of the corresponding group feature fusion neural network for each group according to the number of clients in each group and the state marker information in the request further comprises: the server adds or deletes neurons in the input layer of the group feature fusion neural network for the dynamically added or exited clients in the group according to the state marker information, wherein the neurons are used to receive first intermediate features output by the client feature fusion neural network, the group feature fusion neural network receives output values of the corresponding sub neural networks of all clients in the group and fuses to obtain second intermediate features, and inputs the second intermediate features into the global fusion neural network. 5.The method of claim 4, characterized in that, The server side determines an output result of the global fusion neural network based on the plurality of second intermediate features, and determines a reuse label of the prediction data based on the output result and the prediction data determines a prediction value and a second loss function, and adjusts at least one of the respective sub neural networks, the client feature fusion neural network, the group feature fusion neural network and the global fusion neural network based on back propagation of the second loss function by the server side and the client side, iteratively executes the step of adjusting the neural network based on the back propagation of the first loss function and / or the back propagation of the second loss function until the split neural network converges. 6.The method of claim 5, characterized in that, the server pre-designs a calculation time threshold, and if the time spent by the sub neural network and the client feature fusion neural network corresponding to the client for converging exceeds the calculation time threshold, the server deletes the neurons in the input layer of the group feature fusion neural network corresponding to the client, and sends related indication information of deleting the neurons to the client. If the deleted client re-sends the request or a new client sends a request, the server re-adds the neurons in the group feature fusion neural network input layer for the client, and when the server dynamically generates the client feature fusion neural network and the sub neural network for the corresponding client, at least one of the client feature fusion neural network and the sub neural network is compressed, and the existing sub neural network of the corresponding client is updated with the compressed sub neural network, or If the deleted client re-sends the request or a new client sends a request, the server dynamically generates a compressed client feature fusion neural network for the corresponding client and / or the client generates a compressed sub neural network according to its own metadata.

7. The method of any one of claims 1-6, wherein, Part or all of the clients run in the server.

8. The method of claim 7, wherein, The original data is used as the input data of the sub neural network of the client, and the global fusion sub network of the server outputs the final prediction value.

9. A dynamic construction device based on a split learning neural network, characterized in that, Comprise: At least one processor; And a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

10. A computer readable storage medium, the computer readable storage medium storing a program, when the program is executed by a multi-core processor, the multi-core processor executes the method of any one of claims 1-8.

Images