Multi-level federated collaborative training and inference method for heterogeneous vehicles in the Internet of Vehicles

Abstract

The present invention relates to the field of vehicle networking technology, and specifically discloses a multi-level federated collaborative training and reasoning method for heterogeneous vehicles in the vehicle networking. Without relying on explicit "resource constraints" and "optimization goals", it uses intelligent splitting and cross-layer collaborative training driven by the inherent information flow of a large model. It focuses more on the inherent information flow rules of the model itself, providing a self-consistent and flexible splitting strategy for subsequent multi-level collaborative training and reasoning, and solving the problems of poor computing power adaptation and lack of efficient training when deploying large models on heterogeneous vehicle nodes.

Description

Technical Field

[0001] The present application relates to the field of vehicle networking technology, and specifically discloses a multi-level federated collaborative training and reasoning method for heterogeneous vehicles in vehicle networking. Background Art

[0002] The Internet of Vehicles (IoV) refers to the use of wireless communication technologies, such as on-board sensors, IoV, and cloud computing, to enable real-time data interaction between the vehicle and the external environment, such as infrastructure, other vehicles, and cloud servers, thereby supporting functions such as route planning and driver assistance decisions.

[0003] With the rise of high-precision tasks like autonomous driving and traffic situation prediction, in-vehicle terminals are increasingly demanding high-performance inference capabilities provided by extremely large models (such as Transformers with tens of billions of parameters). Federated learning, a distributed machine learning approach, allows models to be trained on multiple edge devices or local vehicle terminals without uploading data to cloud servers. This not only protects data privacy but also reduces the cost and risk of data transmission. Consequently, it has gained widespread application in the Internet of Vehicles (IoV).

[0004] However, different vehicle models have significant differences in hardware resources, including computing power FLOPS, memory size, and energy consumption, making it difficult to balance real-time and scalability in model deployment. Furthermore, traditional edge computing solutions that employ full offload or simple layering often overlook the potential of local computing power and the diversity of cross-node collaboration.

[0005] The present invention provides a multi-level federated collaborative training and reasoning method for heterogeneous vehicles in the Internet of Vehicles to solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to solve the problems of poor computing power adaptation and lack of efficient training when deploying large models on heterogeneous vehicle nodes.

[0007] To achieve the above objectives, the basic solution of the present invention provides a multi-level federated collaborative training and reasoning method for heterogeneous vehicles in the Internet of Vehicles, including the following steps:

[0008] Step A1: Construct a large model network layer sequence and obtain the high-order mutual information entropy and gradient covariance matrix between adjacent layers of the large model. Based on the obtained information, construct an inter-layer information potential function, find the information breakpoints of the large model by performing discrete differences, and use multi-scale difference detection to identify two optimal information breakpoints. At the two optimal information breakpoints, the large model is split into three sub-models. The three sub-models include: a local sub-model mounted on the vehicle node, an edge sub-model mounted on the edge server, and a cloud sub-model mounted on the cloud.

[0009] Step A2: Each vehicle node performs iterative training based on the divided three-segment sub-model, updates the parameters of the corresponding sub-model, and uploads the updated parameters of the three-segment sub-model to the cloud after completing this round of iterative training. Dynamic weighted aggregation is performed to obtain the aggregated parameters of the local sub-model, the edge sub-model, and the cloud sub-model. The aggregated parameters are then distributed to each vehicle node for the next round of iterative training.

[0010] During the dynamic weighted aggregation process in the cloud, the aggregated parameters are spliced ​​together in layer sequence to form a complete global large model.

[0011] Furthermore, the large model network layer sequence expression is as follows:

[0012] ;

[0013] Where L is the number of network layers of the large model, f L Represents the L-th layer large model network.

[0014] Furthermore, the process of performing discrete differentiation on the inter-layer information potential function includes locating the first-order differential extreme point as the potential split point one, locating the second-order differential extreme point as the potential split point two, and forming a candidate robust information breakpoint group consisting of the potential split point one and the potential split point two.

[0015] Furthermore, the process of identifying two optimal information breakpoints through multi-scale difference detection includes: constructing a comprehensive difference sequence to superimpose the weighted first-order difference and the second-order difference, searching for a group of robust information breakpoints that appear stably at multiple adjacent scales after performing multi-scale convolution to form a candidate set of information breakpoints, and then performing segmented consistency verification to select two optimal information breakpoints from the candidate set of information breakpoints.

[0016] Furthermore, in the implementation process of step A2, the following sub-steps are included:

[0017] Step A201: Use local data samples with sample labels as the first The three-stage forward propagation is performed in sequence in the three-stage sub-model to obtain the predicted label, and the loss between the predicted label and the sample label is obtained by comparing them in the cloud sub-model;

[0018] Step A202: Based on the loss calculated by the cloud sub-model, gradients are propagated backward in the three sub-models in sequence to implement three-stage backpropagation, update the corresponding sub-model parameters, and complete a round of local iteration;

[0019] Step A203: After each vehicle node completes k rounds of local iterations, it uploads the updated parameters of the three-segment model to the cloud, performs dynamic weighted aggregation on the cloud, and distributes the aggregated parameters to each vehicle node to start the next round of training.

[0020] Furthermore, the three-stage forward propagation includes the following steps: local sub-model forward propagation, edge sub-model forward propagation, and cloud sub-model forward propagation. The predicted label is obtained by the cloud sub-model forward propagation process and compared with the sample label to obtain the loss.

[0021] Furthermore, the three-stage back propagation includes the following steps: back propagation of the cloud sub-model and gradient update of the cloud sub-model parameters, back propagation of the edge sub-model and gradient update of the edge sub-model parameters, and back propagation of the local sub-model and gradient update of the local sub-model parameters.

[0022] Furthermore, during the implementation of step A203, the dynamic weighted aggregation of local sub-model parameters is performed to obtain the aggregated parameters:

[0023] ;

[0024] Where L is the number of network layers of the large model, express It is d L dimensional vector, is the αth component of the sum of the parameter values ​​of each vehicle node, α∈{1,…,d L}, The expression is as follows:

[0025] ;

[0026] Where i represents the i-th vehicle node, n is the total number of vehicle nodes, i∈{1,…,n}, is the sum of the parameter values ​​of the same component α at each vehicle node, The expression is as follows:

[0027] ;

[0028] Where, Represents the iteratively updated local sub-model parameter vector of vehicle node i.

[0029] Furthermore, during the implementation of step A203, it also includes dynamic weighted aggregation of edge sub-model parameters and dynamic weighted aggregation of cloud sub-model parameters. The process of dynamic weighted aggregation of edge sub-model parameters and dynamic weighted aggregation of cloud sub-model parameters is the same as the process of dynamic weighted aggregation of local sub-model parameters.

[0030] The principle and effect of this solution are:

[0031] 1. Compared with the existing technology, this invention does not rely on explicit "resource constraints" and "optimization goals". By measuring the high-order mutual information entropy and gradient covariance matrix between different layers of the large model, it constructs an inter-layer information potential function, and uses its discrete difference to identify the two optimal information breakpoints in the network, and finally completes the three-stage splitting, splitting the large model into local sub-models. , edge sub-model , cloud sub-model The three parts not only break away from the dependence on explicit resource constraints or fixed optimization goals, but also can accurately define the boundaries between different layers from the perspective of internal information flow of the model.

[0032] 2. Compared with the existing technology, the present invention also realizes the segmented training and gradient feedback of heterogeneous vehicle nodes in local sub-models, edge sub-models and cloud sub-models through multi-level collaborative forward propagation and backpropagation, completes segmented updates, and further integrates the parameters obtained by each vehicle node on the same sub-model to the cloud, and combines the aggregation results of the three sub-models into a complete large model, thereby taking into account global consistency and local personalization, ensuring that heterogeneous vehicles can not only play the advantages of local data processing in training, but also aggregate the global data through the cloud, forming a complete technical closed loop of "splitting-training-aggregation".

[0033] 3. Compared with the existing technology, the present invention uses intelligent splitting and cross-layer collaborative training driven by the inherent information flow of large models. It focuses more on the inherent information flow rules of the model itself, providing a self-consistent and flexible splitting strategy for subsequent multi-level collaborative training and reasoning, and solves the problems of poor computing power adaptation and lack of efficient training when deploying large models on heterogeneous vehicle nodes. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 A flowchart of a multi-level federated collaborative training and reasoning method for heterogeneous vehicles in the Internet of Vehicles proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0036] In order to further illustrate the technical means and effects adopted by the present invention to achieve the predetermined purpose of the invention, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0037] Multi-level federated collaborative training and reasoning methods for heterogeneous vehicles in the Internet of Vehicles, implementing e.g. Figure 1 As shown, the following steps are included:

[0038] Step A1: Construct a large model network layer sequence and obtain the high-order mutual information entropy and gradient covariance matrix between adjacent layers of the large model. Construct an inter-layer information potential function based on the obtained information. Find the information breakpoints of the large model by performing discrete differences, and use multi-scale difference detection to identify two optimal information breakpoints. Split the large model into three sub-models at the two optimal information breakpoints. The three sub-models include: a local sub-model mounted on the vehicle node, an edge sub-model mounted on the edge server, and a cloud sub-model mounted on the cloud.

[0039] Specifically, the following sub-steps are included:

[0040] Step A101: Construct a large model network layer sequence, calculate the high-order mutual information entropy and gradient covariance matrix between adjacent layers of the large model to quantify the inter-layer dependency, and provide the necessary measurement basis for the subsequent construction of the inter-layer information potential function.

[0041] In this step, the large model network layer Divide into the following sequence:

[0042] ;

[0043] Where L is the number of network layers of the large model, f L Represents the L-th layer large model network.

[0044] When given an input x, the The outputs of layer 1 and layer 0 are:

[0045] ;

[0046] ;

[0047] The calculation process of high-order mutual information entropy between adjacent layers of the large model is as follows:

[0048] Given a ternary mutual information tensor M:

[0049] ;

[0050] Its elements satisfy:

[0051] ;

[0052] Where, Represents the ternary mutual information:

[0053] ;

[0054] This ternary mutual information measures the deeper dependencies or couplings between adjacent layers of the large model. M, a high-order tensor, reflects the multivariate mutual information coupling at different layers of the large model network, providing a core element for the subsequent "inter-layer information potential function."

[0055] The calculation process of the gradient covariance matrix is ​​as follows:

[0056] For the Layer parameters , whose gradient is expressed as For the parameter θ of the u-th layer u , whose gradient is expressed as Therefore, the The conjugate gradient covariance CGC between the layer and the u-th layer is expressed as:

[0057] ;

[0058] Where G is a symmetric positive definite matrix (such as an approximation of the Fisher information matrix).

[0059] The higher the value, the The stronger the coupling between the layer and the u-th layer in gradient update, the The gradient of the layer is more coordinated with that of the u-th layer.

[0060] Step A102: Construct an inter-layer information potential function to integrate high-order mutual information M and gradient coupling CGC, and perform discrete differentiation on the inter-layer information potential function, including locating the first-order differential extreme point as potential split point one, and locating the second-order differential extreme point as potential split point two, and forming a candidate robust information breakpoint group consisting of potential split point one and potential split point two.

[0061] In this embodiment, an interlayer information potential function is given to integrate high-order mutual information M and gradient coupling CGC into a single scalar, so that subsequent "fracture detection" can be performed more intuitively. The expression is as follows:

[0062] ;

[0063] Where α1, α2, α3, and γ are all adjustment coefficients, and v represents the v-th layer large model network. In this formula, the first term reflects the gradient conjugate potential, the second term represents the contribution of the ternary mutual information, and the third term is the regularization compensation for the layer parameter scale.

[0064] After the interlayer information potential function is given, in order to capture the "mutation" or "curvature" of the interlayer information potential function on the layer sequence, the interlayer information potential function is discretely differentiated, including first-order differences:

[0065] ;

[0066] When there is a first-order difference of any layer sequence k , indicating that the information potential Φ decreases significantly between layer sequence k and layer sequence k+1, and layer sequence k is used as a potential information breakpoint splitting point.

[0067] Further includes taking second-order differences:

[0068] ;

[0069] If a second-order difference appears near a certain layer sequence k The extreme value of : maximum or minimum, indicates that the inter-layer information potential function curve has a significant curvature change here, and the layer sequence is used as the potential information breakpoint 2.

[0070] Step A103: Construct a comprehensive difference sequence by weighting the first-order and second-order differences and superimposing them. After performing multi-scale convolution, find a group of robust information breakpoints that consistently appear at multiple adjacent scales to form a candidate set of information breakpoints. Then, perform segmented consistency verification and select the two optimal information breakpoints from the candidate set. These steps further improve detection accuracy and reliability.

[0071] In the process of constructing multi-scale differential signals, a comprehensive differential sequence is constructed to superimpose the weighted first-order and second-order differentials to better capture the dual characteristics of mutation and curvature of information fracture. Expressed as:

[0072] ;

[0073] Where β1 and β2 are adjustable coefficients.

[0074] when If a large jump occurs at a certain location at the same time, D(k) will appear as a large peak / valley value.

[0075] In this embodiment, a family of smoothing kernels or wavelet functions are also introduced (where s represents the scale) to suppress noise and capture fracture morphologies at different scales.

[0076] In summary, through discrete sequence The expression for multi-scale convolution is as follows:

[0077] ;

[0078] At different scales s, multiple smoothed difference sequences are obtained As the scale s increases, small scale s is more sensitive to subtle jumps, and large scale s is more likely to capture global or macroscopic breaks. In the multi-scale convolution corresponding to each scale s In the above example, we detect the local extreme point set Ω s , and then track the extreme points on the scale axis (similar to "scale space extreme value tracking"), find the robust information breakpoint group that appears stably on multiple adjacent scales, and form a candidate set of information breakpoints In this embodiment, only those points that are prominent at multiple scales are considered as robust information breakpoints, and multi-scale extreme value detection is performed in this way.

[0079] After obtaining a set of candidate information breakpoints, we must select two points that best segment the large model network into three self-consistent submodels. In this embodiment, a segment consistency comprehensive metric function C(i, j) is constructed to measure the intrinsic coherence and cross-segment differences of the three segments in terms of information potential. This is used to evaluate the quality of the three-segment partitioning corresponding to any two robust information breakpoints (i, j), where i represents robust information breakpoint 1 and j represents robust information breakpoint 2.

[0080] Specifically, segment consistency verification includes: intra-segment consistency verification and cross-segment consistency verification. Before segment consistency verification, a set of layer sequences within each segment is pre-established. , Γ1={1,…,i} represents the first segment, Γ2={i+1,…,j} represents the second segment, and Γ3={j+1,…,L} represents the third segment.

[0081] The intra-segment consistency verification is used to verify the consistency of each segment, including constructing the intra-segment consistency verification function based on the established layer sequence set as follows:

[0082] ;

[0083] In the formula, the first term represents the average level of information potential of all layers in the segment, and the second term represents the difference between layers in the segment. If the distribution is concentrated, it means that the intra-segment consistency Intra(Γ) of the segment is high.

[0084] Cross-segment consistency verification is used to verify the consistency between each segment. When there are three segments, their layer sequence sets are marked as Γ1, Γ2, and Γ3 respectively, and the cross-segment consistency verification function is constructed as follows:

[0085] ;

[0086] Where μ(Γ) represents the average potential value of a certain segment. The larger the difference between different segments, the more significant the fracture.

[0087] When there are any two robust information breakpoints (i, j) that divide the large model into three segments:

[0088] Γ1={1,…,i}, Γ2={i+1,…,j}, Γ3={j+1,…,L};

[0089] A piecewise consistency comprehensive metric function C(i, j) is constructed to evaluate the quality of the three-segment partitioning corresponding to any two selected robust information breakpoints (i, j). The expression of the piecewise consistency comprehensive metric function C(i, j) is as follows:

[0090] C(i,j)=λ1[Intra(Γ1)+Intra(Γ2)+Intra(Γ3)]+λ2Inter(Γ1,Γ2,Γ3);

[0091] Where λ1 and λ2 are used to balance the consistency within a segment and the difference between segments.

[0092] When C(i,j) is larger, it means that after segmentation using the selected robust information breakpoint (i,j), the segments are more compact and the segments are more distinct.

[0093] Based on the established comprehensive metric function, the two robust information breakpoints that maximize the segment consistency comprehensive metric function C(i,j) are selected as the optimal information breakpoints according to the following formula:

[0094] ;

[0095] Step A104: Using the two selected optimal information breakpoints :As the actual large model segmentation point, the large model is decomposed into three sub-models: local sub-model , edge sub-model and cloud sub-model , the expressions are as follows:

[0096] ;

[0097] ;

[0098] ;

[0099] Local sub-model The first few layers: 1st to The layer is used to preliminarily process the data input by the original sensor. According to the fracture of the information potential function between layers, the front There are relatively consistent high-order correlations within the layer, but with the subsequent layers There is an obvious information gap between +1, so it is used as a local sub-model.

[0100] Edge submodel Contains the most active middle layers in the large model network: +1 layer to In the Internet of Vehicles scenario, the edge sub-model composed of edge servers has medium to high computing power and can bear the large computing load of this part. A significant information potential drop occurs again, which is conducive to separating the edge sub-model from the cloud sub-model.

[0101] Cloud sub-model is the final set of high-level or output layers: From layer +1 to layer L, within this layer range, the large model network may perform global semantic aggregation or decision output. After further weakening, it can be placed in the cloud for larger-scale model maintenance and calculation.

[0102] Step A2: Each vehicle node performs iterative training based on the divided three-segment sub-model, updates the parameters of the corresponding sub-model, and uploads the updated parameters of the three-segment sub-model to the cloud after completing this round of iterative training. Dynamic weighted aggregation is performed to obtain the aggregated parameters of the local sub-model, the edge sub-model, and the cloud sub-model. The aggregated parameters are then distributed to each vehicle node for the next round of iterative training.

[0103] During the dynamic weighted aggregation process in the cloud, the aggregated parameters are spliced ​​together in layer sequence to form a complete global large model.

[0104] Specifically, the following steps are included:

[0105] Step A201: Take the sample labels Local data sample x i As the input of the i-th vehicle node, a three-stage forward propagation is performed in sequence in the three-stage sub-model to obtain the predicted label, and the loss between the predicted label and the sample label is obtained by comparison in the cloud sub-model, i∈{1,…,n}.

[0106] The three-stage forward propagation specifically includes the following steps:

[0107] Local sub-model forward propagation: The local sub-model carried by each vehicle node receives the input x given by the sensor i , the vehicle node saves the parameters in the local sub-model , corresponding to the local sub-model Parameters of each layer When given input x i When the local output activation function is calculated locally, the local output activation function is calculated locally according to the formula :

[0108] ;

[0109] And send it to the edge server after calculation.

[0110] Edge sub-model forward propagation: by edge sub-model Receive local output activation function from local sub-model , edge submodel Save the parameter θ E , calculate the intermediate activation function as follows:

[0111] ;

[0112] After the calculation is completed, the intermediate activation function Send sub-model to the cloud.

[0113] Cloud sub-model forward propagation: Cloud sub-model The parameter is denoted as θ C , received by the cloud sub-model , calculate according to the following formula, and use the calculation result as the predicted label y i :

[0114] ;

[0115] Loss calculation: After obtaining the predicted label, calculate the loss according to the following formula :

[0116] ;

[0117] Where, is the sample label.

[0118] At this point, the single vehicle completes a forward propagation from the local sub-model to the edge sub-model and finally to the cloud sub-model, and obtains the predicted label y in the cloud sub-model i and losses .

[0119] Step A202: Calculate the loss based on the cloud sub-model , gradients are propagated backward in the three-stage sub-model in turn to realize three-stage back propagation, update the corresponding sub-model parameters, and complete a round of local iteration.

[0120] The loss of the i-th vehicle based on cloud computing , backward in sequence: the cloud sub-model to the edge sub-model, and finally to the local sub-model to perform gradient backpropagation and update the corresponding model parameters.

[0121] The three-stage back propagation specifically includes the following steps:

[0122] Back propagation and update of cloud sub-model: Back propagation and update of cloud sub-model parameters θ C , and perform backpropagation directly in the cloud sub-model as follows:

[0123] ;

[0124] And update the parameters as follows:

[0125] ;

[0126] Where η is the learning rate.

[0127] At the same time, the intermediate activation function is calculated and passed back to the edge sub-model Gradient:

[0128] ;

[0129] Back propagation and update of edge sub-model: The edge sub-model receives Then, the parameter θ is calculated as follows: E Perform gradient calculation:

[0130] ;

[0131] And update the parameters as follows:

[0132] ;

[0133] Pass gradients back to the local sub-model:

[0134] ;

[0135] Back propagation and update of local sub-model: received by the local sub-model Then, adjust the parameters according to the formula Perform gradient calculation:

[0136] ;

[0137] And update the parameters as follows:

[0138] ;

[0139] At this point, vehicle node i completes the three-segment parameter A local update of .

[0140] Step A203: After each vehicle node completes k rounds of local iterations, it uploads the updated parameters of the three-segment model to the cloud, performs dynamic weighted aggregation on the cloud, and distributes the aggregated parameters to each vehicle node to start the next round of training.

[0141] In the forward propagation and backpropagation, each vehicle node i∈{1,…,n} has completed several rounds of local iterations and obtained its iteratively updated three-stage model parameters:

[0142] ;

[0143] Where, is the iteratively updated local sub-model parameter of vehicle node i, is the edge sub-model parameter after iterative update, are the cloud sub-model parameters after iterative update.

[0144] Due to the adoption of the “three-segment split”, it is now necessary to aggregate the parameters of different vehicle nodes on the same sub-model segment and finally obtain a complete large model θ final The specific process includes the following:

[0145] Dynamic weighted aggregation of local sub-model parameters: Let , represents the iteratively updated local sub-model parameter vector of vehicle node i, d L Represents the dimension, the vector contains the dimensions from the 1st layer to the All weights obtained by layer training on the vehicle node. When there are n vehicles in total, these vectors are collected as:

[0146] ;

[0147] The parameter values ​​of the same component α at each vehicle node are added together to perform component-by-component summation of the local sub-model parameters. The expression is as follows:

[0148] ;

[0149] Where, For parameters The value of the αth component, α∈{1,…,d L}.

[0150] Based on the component-by-component summation formula of the local sub-model parameters, the sum of each component Divide by the total number of vehicle nodes n to construct a new vector , generates the local sub-model global vector, whose αth component The expression is as follows:

[0151] ;

[0152] Finally, the dynamic weighted aggregation of the local sub-model parameters is performed to obtain the aggregated parameters of the local sub-model in a global sense:

[0153] ;

[0154] Dynamic weighted aggregation of marginal sub-model parameters: Let Represents the iteratively updated edge sub-model parameter vector of vehicle node i, corresponding to the layer sequence When there are n vehicles in total, these vectors are collected as:

[0155] ;

[0156] The parameter values ​​of the same component β at each vehicle node are added together to perform component-by-component summation of the local sub-model parameters. The expression is as follows:

[0157] ;

[0158] Where, For parameters The value of the βth component, β∈{1,…,d E}.

[0159] Based on the component-by-component summation formula of the edge sub-model parameters, the sum of each component Divide by the total number of vehicle nodes n to construct a new vector , generates the edge sub-model global vector, whose βth component The expression is as follows:

[0160] ;

[0161] Finally, the dynamic weighted aggregation of the edge sub-model parameters is performed to obtain the aggregated parameters of the edge sub-model in a global sense:

[0162] ;

[0163] Since the “edge sub-models” may be deployed on different edge servers or relay nodes in the present invention, they are uniformly regarded as the “intermediate sub-models” owned by the vehicle side in the federation aggregation stage and aggregated in the cloud. These are globally shared intermediate sub-model parameters. During the aggregation phase, each vehicle node reports the edge sub-model parameters transferred via the edge server to the cloud, which then collects all edge sub-model parameters for merging.

[0164] Dynamic weighted aggregation of cloud sub-model parameters: Let Represents the cloud sub-model parameter vector after iterative update of vehicle node i, corresponding to the layer sequence When there are n vehicles in total, these vectors are collected as:

[0165] ;

[0166] The parameter values ​​of the same component γ at each vehicle node are added together to calculate the component-by-component sum of the cloud sub-model parameters. The expression is as follows:

[0167] ;

[0168] Where, For parameters The value of the γth component, γ∈{1,…,d C}.

[0169] Based on the component-by-component summation formula of the cloud sub-model parameters, the sum of each component Divide by the total number of vehicle nodes n to construct a new vector , generating the global vector of the cloud sub-model, whose γth component The expression is as follows:

[0170] ;

[0171] Finally, the global aggregation of the cloud sub-model parameters is performed to obtain the aggregated parameters of the cloud sub-model in a global sense:

[0172] ;

[0173] After completing the aggregation of the three parameters, we get From the perspective of neural network structure, the local sub-model parameters correspond to the layer , the edge sub-model parameters correspond to the layer , the cloud sub-model parameters correspond to the layer .

[0174] Therefore, we only need to "join" these three parameters in layer sequence to form a complete global model: .

[0175] Without relying on explicit "resource constraints" and "optimization goals", this invention constructs an inter-layer information potential function by measuring the high-order mutual information entropy and gradient covariance matrix between different layers of the large model, and uses its discrete difference to identify the two optimal information breakpoints in the network, and finally completes the three-stage splitting, splitting the large model into local sub-models. , edge sub-model , cloud sub-model The three parts not only break away from the dependence on explicit resource constraints or fixed optimization goals, but also can accurately delineate the boundaries of different layers from the perspective of the internal information flow of the model. The present invention also realizes the segmented training and gradient feedback of heterogeneous vehicle nodes in local sub-models, edge sub-models and cloud sub-models through multi-level collaborative forward propagation and backpropagation, completes segmented updates, and further integrates the parameters obtained by each vehicle node on the same sub-model to the cloud, and combines the aggregation results of the three sub-models into a complete large model, thereby taking into account global consistency and local personalization, ensuring that heterogeneous vehicles can not only play the advantages of local data processing in training, but also aggregate the global situation through the cloud, forming a complete technical closed loop of "splitting-training-aggregation".

[0176] In summary, the present invention uses intelligent splitting and cross-layer collaborative training driven by the inherent information flow of the large model, and focuses more on the inherent information flow rules of the model itself to provide a self-consistent and flexible splitting strategy for subsequent multi-level collaborative training and reasoning, solving the problems of computing power adaptation and efficient training for the deployment of large models on heterogeneous vehicle nodes, and providing a self-consistent and efficient technical solution for the application of distributed large models in heterogeneous vehicle scenarios.

[0177] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any person skilled in the art can make some changes or modifications to equivalent embodiments using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A multi-level federated collaborative training and reasoning method for heterogeneous vehicles in the Internet of Vehicles, characterized by: The steps include: Step A1: Construct a large model network layer sequence and obtain the high-order mutual information entropy and gradient covariance matrix between adjacent layers of the large model. Based on the obtained information, construct an inter-layer information potential function, find the information breakpoints of the large model by performing discrete differences, and use multi-scale difference detection to identify two optimal information breakpoints. At the two optimal information breakpoints, the large model is split into three sub-models. The three sub-models include: a local sub-model mounted on the vehicle node, an edge sub-model mounted on the edge server, and a cloud sub-model mounted on the cloud. Step A2: Each vehicle node performs iterative training based on the divided three-segment sub-model, updates the parameters of the corresponding sub-model, and uploads the updated parameters of the three-segment sub-model to the cloud after completing this round of iterative training. Dynamic weighted aggregation is performed to obtain the aggregated parameters of the local sub-model, the edge sub-model, and the cloud sub-model. The aggregated parameters are then distributed to each vehicle node for the next round of iterative training. During the dynamic weighted aggregation process in the cloud, the aggregated parameters are spliced ​​together in layer sequence to form a complete global large model; The process of performing discrete difference on the inter-layer information potential function includes locating the first-order difference extreme value point as the potential split point 1, locating the second-order difference extreme value point as the potential split point 2, and forming a candidate robust information breakpoint group from the potential split point 1 and the potential split point 2; The process of identifying two optimal information breakpoints through multi-scale difference detection includes: constructing a comprehensive difference sequence, superimposing the weighted first-order difference and the second-order difference, searching for a group of robust information breakpoints that appear stably at multiple adjacent scales after performing multi-scale convolution to form a candidate set of information breakpoints, and then performing segmented consistency verification to select two optimal information breakpoints from the candidate set of information breakpoints.

2. The multi-level federated collaborative training and reasoning method for heterogeneous vehicles in the Internet of Vehicles according to claim 1 is characterized in that: The large model network layer sequence expression is as follows: ； Where L is the number of network layers of the large model, f L Represents the L-th layer large model network.

3. The multi-level federated collaborative training and reasoning method for heterogeneous vehicles in the Internet of Vehicles according to claim 1 is characterized in that: The implementation of step A2 includes the following sub-steps: Step A201: Use local data samples with sample labels as the first The three-stage forward propagation is performed in sequence in the three-stage sub-model to obtain the predicted label, and the loss between the predicted label and the sample label is obtained by comparing them in the cloud sub-model; Step A202: Based on the loss calculated by the cloud sub-model, gradients are propagated backward in the three sub-models in sequence to implement three-stage backpropagation, update the corresponding sub-model parameters, and complete a round of local iteration; Step A203: After each vehicle node completes k rounds of local iterations, it uploads the updated parameters of the three-segment model to the cloud, performs dynamic weighted aggregation on the cloud, and distributes the aggregated parameters to each vehicle node to start the next round of training.

4. The multi-level federated collaborative training and reasoning method for heterogeneous vehicles in the Internet of Vehicles according to claim 3 is characterized in that: The three-stage forward propagation includes: local sub-model forward propagation, edge sub-model forward propagation and cloud sub-model forward propagation in sequence. The predicted label is obtained by the cloud sub-model forward propagation process and compared with the sample label to obtain the loss.

5. The multi-level federated collaborative training and reasoning method for heterogeneous vehicles in the Internet of Vehicles according to claim 4 is characterized in that: The three-stage back propagation includes the following steps: back propagation of the cloud sub-model and gradient update of the cloud sub-model parameters, back propagation of the edge sub-model and gradient update of the edge sub-model parameters, and back propagation of the local sub-model and gradient update of the local sub-model parameters.

6. The multi-level federated collaborative training and reasoning method for heterogeneous vehicles in the Internet of Vehicles according to claim 3 is characterized in that: The implementation process of step A203 includes dynamic weighted aggregation of local sub-model parameters to obtain aggregated parameters: ； Where L is the number of network layers of the large model, express It is d L dimensional vector, is the αth component of the sum of the parameter values ​​of each vehicle node, α∈{1,…,d L }, The expression is as follows: ； Where i represents the i-th vehicle node, n is the total number of vehicle nodes, i∈{1,…,n}, is the sum of the parameter values ​​of the same component α at each vehicle node, The expression is as follows: ； Where, Represents the iteratively updated local sub-model parameter vector of vehicle node i.

7. The multi-level federated collaborative training and reasoning method for heterogeneous vehicles in the Internet of Vehicles according to claim 6 is characterized in that: The implementation process of step A203 also includes dynamic weighted aggregation of edge sub-model parameters and dynamic weighted aggregation of cloud sub-model parameters. The process of dynamic weighted aggregation of edge sub-model parameters and dynamic weighted aggregation of cloud sub-model parameters is the same as the process of dynamic weighted aggregation of local sub-model parameters.

Images