A data sharing exchange method based on deep learning

Abstract

The application discloses a kind of data sharing exchange methods based on deep learning, it is related to computer technology field, the dynamic evaluation of the present application based on state metadata, real-time traffic event, vehicle space-time state and equipment quality are mapped into value evaluation score together, make client participate in decision-making real-time update with event evolution, to reduce the omission of key data under the static trajectory similarity in sudden scene;Road network topology coding and space-time fusion mechanism of event guide are introduced in value evaluation network, so that the influence of event can cover upstream and downstream road sections along the topological relationship, and is associated with time difference, and then improve the identification ability of key path vehicle and scheduling consistency, in addition, positioning accuracy, sensor effectiveness, missing rate and other quality information are included in evaluation input, and missing / verification processing and parameter update effectiveness checking mechanism are introduced in training scheduling and aggregation link, so that abnormal client update is inhibited or removed, to improve the stability of collaborative training process.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a data sharing and exchange method based on deep learning. Background Technology

[0002] In intelligent transportation and vehicle-to-everything (V2X) applications, vehicles and roadside / cloud nodes continuously generate multi-source heterogeneous sensing data. Due to privacy protection, bandwidth, and latency constraints, existing technologies often adopt distributed collaborative learning or parameter exchange to achieve cross-entity data sharing. That is, each node trains a model locally and exchanges model parameters, feature representations, or statistics to support tasks such as traffic flow prediction and congestion warning. Related solutions need to select some nodes from a large number of candidate vehicles to participate in each round of collaboration in order to reduce communication overhead and improve model convergence efficiency. Existing selection mechanisms mostly rely on indicators such as the similarity of node data distribution or behavioral trajectories and gradient consistency for screening or cluster sampling.

[0003] However, urban traffic conditions are highly dynamic and unpredictable. When accidents, temporary traffic control measures, or localized congestion occur, the value of data can shift rapidly in time and space. Vehicles not located in the core area of ​​the incident but on key upstream diversion paths may have a higher contribution to congestion spread prediction due to their perceived traffic changes, but they may be underestimated or excluded due to static similarity mismatch. At the same time, vehicle sensors and positioning modules may experience quality fluctuations due to occlusion, contamination, calibration drift, etc., resulting in significant differences in the reliability of observations at the same location. If the selection and exchange strategies do not incorporate data quality and timeliness into a unified quantification, it will cause a mismatch between the investment of computing and communication resources and the actual data contribution, thereby affecting the stability and prediction accuracy of the collaborative model. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] This invention provides a data sharing and exchange method based on deep learning to solve the problem of difficulty in handling sudden events and equipment quality fluctuations when filtering clients based on similar historical trajectories.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, embodiments of the present invention provide a data sharing and exchange method based on deep learning, applied to a federated learning system comprising a central server and multiple vehicle clients, including:

[0008] Step S1: The central server obtains the status metadata reported by each vehicle client, inputs the status metadata into the pre-trained client data value dynamic evaluation network, and outputs the value evaluation score corresponding to each vehicle client.

[0009] Step S2: Under the constraints of the given communication resource budget and computing resource budget, the central server determines the set of vehicle clients participating in this round of training based on the value assessment score, and allocates resource quotas to each vehicle client in the set.

[0010] Step S3: Under the resource quota constraints, the vehicle clients within the set perform local model training and upload model parameter updates to the central server.

[0011] Step S4: The central server aggregates the model parameter updates to update the global model.

[0012] As a preferred embodiment of the deep learning-based data sharing and exchange method of the present invention, the state metadata includes at least: vehicle dynamic spatiotemporal information, real-time traffic event information associated with the dynamic spatiotemporal information, and vehicle equipment status quality information; wherein, the vehicle dynamic spatiotemporal information includes a position sequence and / or speed sequence within the most recent preset time window, the real-time traffic event information includes the event type, the road network element identifier corresponding to the event location, and the event timestamp, and the equipment status quality information includes positioning accuracy indicators, sensor effectiveness indicators, and / or data missing rate indicators.

[0013] As a preferred embodiment of the deep learning-based data sharing and exchange method of the present invention, the client data value dynamic evaluation network includes at least: a graph neural network layer for encoding the traffic network topology map to output graph encoding results, a time-series coding layer for encoding the location sequence and / or speed sequence to output time-series coding results, an attention fusion layer for fusing the graph encoding results, the real-time traffic event information and the time-series coding results, and a fully connected layer for outputting the value evaluation score based on the fusion results.

[0014] As a preferred embodiment of the deep learning-based data sharing and exchange method of the present invention, wherein: the pre-training of the client data value dynamic evaluation network is completed based on historical federated learning round samples, the round samples include at least state metadata and corresponding contribution labels; the contribution labels are obtained by the global model performance gain index caused by the participation of the vehicle client in the training in the corresponding round, the performance gain index includes the reduction in validation set prediction error and / or the reduction in validation set loss.

[0015] As a preferred embodiment of the deep learning-based data sharing and exchange method of the present invention, when the central server determines the set of vehicle clients participating in the current training round and allocates resource quotas based on the value assessment score, the process includes: obtaining the resource cost estimate of each vehicle client; calculating the scheduling priority based on the value assessment score and the resource cost estimate; sorting the vehicles according to the scheduling priority and selecting the top few vehicle clients to form the set under the constraints of the communication resource budget and the computing resource budget, and determining the communication resource quota and computing resource quota for each vehicle client in the set.

[0016] As a preferred embodiment of the data sharing and exchange method based on deep learning described in this invention, the communication resource quota includes the maximum number of uplink transmission bytes and / or the maximum number of uploads and / or the number of uplink time slots, and the computing resource quota includes the upper limit of the number of local training rounds and / or the upper limit of the number of local training batches and / or the upper limit of the local training duration.

[0017] As a preferred embodiment of the deep learning-based data sharing and exchange method described in this invention, the resource overhead estimation includes at least uplink transmission latency estimation and local training latency estimation. The uplink transmission latency estimation is determined by the available uplink rate of the vehicle client and the amount of parameters to be uploaded. The amount of parameters to be uploaded is the amount of model parameter update data generated by the vehicle client after the current round of local training and to be uploaded. The local training latency estimation is determined by the vehicle client's computing power index and the computing resource quota. The computing power index is a representation of the vehicle client's ability to complete the number of local training steps within a preset time.

[0018] As a preferred embodiment of the deep learning-based data sharing and exchange method described in this invention, the central server aggregates the model parameter updates by weighting the parameter updates according to the value assessment score corresponding to the vehicle client. The weight of the weighted aggregation is determined by the value assessment score and normalized. Furthermore, the central server does not receive parameter updates from unscheduled vehicle clients.

[0019] In a second aspect, embodiments of the present invention provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements any step of the data sharing and exchange method based on deep learning as described in the first aspect of the present invention.

[0020] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the data sharing and exchange method based on deep learning as described in the first aspect of the present invention.

[0021] Through the above technical solution, the present invention can achieve at least the following beneficial effects:

[0022] To address the issue of rapid data value shift caused by sudden traffic incidents, this invention proposes a dynamic evaluation based on state metadata. It maps real-time traffic events, vehicle spatiotemporal status, and equipment quality together into a value assessment score, enabling client-side decision-making to be updated in real time as the event evolves. This reduces the omission of key data due to static trajectory similarity in sudden incident scenarios.

[0023] To address the issue that vehicles on critical upstream diversion paths, though not located in the core area of ​​an event, are more critical for predicting congestion spread, this invention introduces a spatiotemporal fusion mechanism of road network topology coding and event guidance into the value assessment network. This enables the impact of events to cover upstream and downstream road segments along the topological relationship and to be correlated with time differences, thereby improving the ability to identify vehicles on critical paths and the consistency of scheduling.

[0024] To address the problem of unstable data quality and low-quality updates that disturb the global model due to fluctuations in vehicle sensor status, this invention incorporates quality information such as positioning accuracy, sensor effectiveness, and missing rate into the evaluation input. Furthermore, it introduces missing / verification processing and parameter update validity verification mechanisms in the training scheduling and aggregation stages to suppress or eliminate abnormal client updates, thereby improving the stability of the collaborative training process.

[0025] To address the issues of limited communication and computing resources and the mismatch between traditional allocation and data value, this invention uses value assessment scores and resource cost estimates to construct scheduling priorities under budget constraints, and determines the participation set and communication / computing quotas accordingly. This makes resource input monotonically correlated with expected contribution, reducing resource crowding and single-round latency fluctuations caused by high-cost, low-return participation.

[0026] To address the issues of unstable participation and strong heterogeneity in updates that arise in federated learning within the vehicle-to-everything (V2X) mobile environment, this invention organizes evaluation, selection, quota allocation, training, uploading, and aggregation into a round-based closed-loop process. Updates are not received by clients that are not scheduled, thus clarifying the participation boundaries, defining resource constraints, and ensuring controllable update sources for each training round, thereby enhancing the controllability and consistency of the global model update process. Attached Figure Description

[0027] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation on the scope of this application.

[0028] Figure 1 This is a flowchart of the data sharing and exchange method in the embodiment. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0030] All terms used in this application (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0031] Example 1:

[0032] like Figure 1 As shown, this application proposes a data sharing and exchange method based on deep learning, applied to a federated learning system containing a central server and multiple vehicle clients, including the following steps:

[0033] Step S1: The central server obtains the status metadata reported by each vehicle client, inputs the status metadata into the pre-trained client data value dynamic evaluation network, and outputs the value evaluation score corresponding to each vehicle client.

[0034] The status metadata is generated and reported by the vehicle client before or at the start of this round of federated learning. The status metadata is encapsulated in a unified field format and carries the generation timestamp and vehicle client identifier. The status metadata includes dynamic spatiotemporal information fields, real-time traffic event information fields, and equipment status quality information fields. The dynamic spatiotemporal information fields and real-time traffic event information fields are associated through road network element identifiers.

[0035] The value assessment score is a normalized scalar, representing the expected contribution of the vehicle client under the current time, current task, and current event conditions. The value assessment score is directly output by the dynamic evaluation network of client data value and mapped to a preset finite interval according to a preset normalization rule. When the vehicle client's state metadata has missing fields or field validation failures, the central server performs default filling for the missing fields and puts a missing identifier in the state metadata, and also inputs the missing identifier into the dynamic evaluation network of client data value. When the vehicle client reports field validation failures for multiple consecutive rounds, the central server sets the vehicle client's value assessment score to the lowest valid value and does not schedule the vehicle client to participate in training.

[0036] Step S2: Under the constraints of the given communication resource budget and computing resource budget, the central server determines the set of vehicle clients participating in this round of training based on the value assessment score, and allocates resource quotas to each vehicle client in the set.

[0037] The communication resource budget is the total uplink transmission resource allowed in this round of federated learning, and the computational resource budget is the total local training resource allowed in this round of federated learning. The resource quota includes at least the communication resource quota and the computational resource quota. The communication resource quota is used to constrain the parameter update upload behavior of the vehicle client, and the computational resource quota is used to constrain the local training behavior of the vehicle client. When the vehicle client reaches the computational resource quota limit during this round of training, the vehicle client terminates local training and enters the upload phase. When the vehicle client reaches the communication resource quota limit during the upload process, the vehicle client stops uploading and only uploads the generated complete parameter update fragment.

[0038] Step S3: Under resource quota constraints, the vehicle clients within the set perform local model training and upload model parameter updates to the central server;

[0039] Step S4: The central server aggregates the model parameter updates to update the global model;

[0040] The central server performs validity checks on the received model parameter updates. The validity checks include at least parameter update integrity checks and numerical range checks. When a parameter update fails the validity checks, the central server discards the corresponding parameter update and sets the corresponding vehicle client's value assessment score for this round to the lowest valid value. The aggregation adopts a weighted aggregation method. The aggregation weight is determined by the value assessment score of the corresponding vehicle client and is normalized. Unscheduled vehicle clients do not participate in this round of aggregation.

[0041] In this embodiment, the status metadata includes at least: vehicle dynamic spatiotemporal information, real-time traffic event information associated with the dynamic spatiotemporal information, and vehicle equipment status quality information; the dynamic spatiotemporal information field includes at least a continuous observation sequence within the most recent preset time window, the continuous observation sequence is arranged in chronological order and includes position observation and motion state observation; the real-time traffic event information field consists of event records, the event record includes at least the event type, the time of event occurrence, and the road network element identifier corresponding to the event, and when multiple event records exist at the same time, the central server filters the event records associated with the vehicle client based on the topological distance between the road network element identifier where the vehicle client is located and the road network element identifier corresponding to the event; when no event record associated with the vehicle client is obtained, the real-time traffic event information field is set to an empty event identifier and carries an empty event timestamp. The vehicle dynamic spatiotemporal information includes a position sequence and / or speed sequence within the most recent preset time window, the real-time traffic event information includes the event type, the road network element identifier corresponding to the event occurrence location, and the event timestamp, and the equipment status quality information includes positioning accuracy indicators, sensor effectiveness indicators, and / or data missing rate indicators;

[0042] The positioning accuracy index is calculated by the vehicle client based on the quality field of positioning observation and reported along with the status metadata; the sensor effectiveness index is generated by the vehicle client based on the sensor self-test results and data availability status; the data missing rate index is calculated by the vehicle client based on the ratio of the number of valid samples to the number of required samples within a preset time window; when the sensor effectiveness index indicates sensor failure or the data missing rate index indicates severe missing data, the central server sets the value assessment score of the corresponding vehicle client to the lowest valid value and does not include the vehicle client in the set of vehicle clients participating in the training in this round.

[0043] In this embodiment, the client data value dynamic evaluation network includes at least: a graph neural network layer for encoding the traffic network topology map to output graph encoding results, a time-series coding layer for encoding the position sequence and / or speed sequence to output time-series coding results, an attention fusion layer for fusing the graph encoding results, real-time traffic event information and time-series coding results, and a fully connected layer for outputting a value evaluation score based on the fusion results.

[0044] The traffic network topology map consists of a set of network elements and a set of connections. Network elements include road segment elements and intersection elements, and connections include adjacent connectivity and directional relationships. The central server generates network element features for each network element. These features include at least road grade information and traffic capacity information, and are used as the initial input features for the graph neural network layers. The location observations of vehicle clients are mapped to unique network element identifiers through map matching rules. The network element identifiers corresponding to the location of events in real-time traffic event information maintain the same identifier space as the network element identifiers mapped by the vehicle clients.

[0045] The attention fusion layer receives the road network element representation output by the graph neural network layer, the trajectory temporal representation output by the temporal coding layer, and the event representation of real-time traffic event information, and calculates the correlation weight between the event representation and the road network element representation and the trajectory temporal representation. The attention fusion layer performs weighted aggregation of the road network element representation, trajectory temporal representation and event representation based on the correlation weight to generate a spatiotemporal fusion representation. When the real-time traffic event information field is an empty event identifier, the attention fusion layer assigns the lowest correlation weight to the event representation and performs weighted aggregation based only on the road network element representation and the trajectory temporal representation.

[0046] In one implementation, the road network map encoding results, real-time traffic event information, and trajectory temporal encoding results are organized into event-guided temporal attention in the attention fusion layer. Before entering the temporal attention layer, graph attention modulation of event propagation along the topology is introduced, ensuring that the event's impact covers the upstream and downstream road segments of the event point and is correlated with the event's time difference. Specifically:

[0047] When real-time traffic event information is encoded into event embedding vectors, it can be obtained by linear mapping:

[0048] ,

[0049] in, Represents an event embedding vector; The event embedding mapping matrix represents the event embedding. This represents an event feature vector consisting of the event type, the road network element identifier corresponding to the event location, and the event timestamp. This represents the event embedding bias vector.

[0050] When the graph neural network layer outputs road network elements Graph encoding vector Then, graph attention with event modulation terms can be introduced onto the adjacent edges of the road network to obtain the event-modulated graph encoding. :

[0051] ,

[0052] in, Representing road network elements Modulation factor of relative events; Indicates the topological distance attenuation coefficient; Representing road network elements Road network elements at the event's endpoint The topological hop count; Indicates the event time decay coefficient; Indicates the reference time corresponding to the current round; Indicates the event timestamp; This indicates the road network element identifier corresponding to the location where the event occurred.

[0053] ,

[0054] in, Representing adjacent road network elements Road network elements Attention score; Representing road network elements Modulation factor; Represents the graph attention scoring vector; Represents the characteristic linear transformation matrix of the graph; Representing road network elements Graph encoding vector; Representing road network elements Graph encoding vector; This indicates vector concatenation.

[0055] ,

[0056] in, This represents the normalized graph attention coefficients; Representing road network elements The set of adjacent elements in the road network topology; Represents the road network elements after event modulation Graph encoding vector; This represents a non-linear activation function.

[0057] When the sequential coding layer outputs the first time window within the preset time window The time series vector of each observation and its timestamp At this time, dot product attention of event query-trajectory key values ​​can be used, combined with time decay, to make it closer to the target. The observation weight is greater:

[0058] ,

[0059] in, Represents the attention query vector; Represents the query mapping matrix; Road network elements indicating the location of the event Event modulation graph encoding vector; Represents the query bias vector; Indicates the first The key vector of each observation; Represents the key mapping matrix; Indicates the first The time-series vector of each observation; Indicates the first A vector of observed values; Represents a value mapping matrix.

[0060] ,

[0061] in, Indicates the first Each observation relative reference time Time decay factor; Indicates the timing decay coefficient; Indicates the first Each observation timestamp.

[0062] ,

[0063] in, Indicates the first Temporal attention coefficients for each observation; This indicates the number of observations within a preset time window; Indicates the dimension of the key vector; This represents the trajectory context vector obtained by the event guidance.

[0064] When using multi-head attention to enhance expressive power, the outputs of each head can be concatenated and linearly mapped to the output of the fusion layer:

[0065] ,

[0066] in, This represents the output vector of the attention fusion layer; This represents the multi-head output mapping matrix; Indicates the first The context vector of each attention head; Indicates the number of attention heads; This represents the multi-head output bias vector. Can be with and The concatenated data is input into a fully connected layer to output a value assessment score.

[0067] Specifically, the attention fusion layer, guided by events and road network topology, selectively converges trajectory temporal codes, establishing a correspondence between the fusion results and the location, timing, and type of traffic events encountered by vehicles. Event embedding unifies event type and temporal information into a vector space, facilitating similarity calculations with graph coding and temporal coding. Graph attention propagates event influence between adjacent road segments and modulates it using topological hop count and event time difference, making the scope of event influence vary with spatial distance and time interval, reducing interference from features of road segments unrelated to the event on the fusion results. Temporal attention uses event-related representations as queries, weighting trajectory observations while adding a time decay factor, giving higher weight to observations closer to the current moment while retaining the ability to aggregate long-term information such as speed change trends. The multi-head structure simultaneously learns event-trajectory association patterns in different subspaces, making the fusion representation more suitable for the subsequent fully connected layer to output value evaluation scores, and forming a consistent data value-driven logic with the resource scheduling process.

[0068] In this embodiment, the pre-training of the client data value dynamic evaluation network is completed based on historical federated learning round samples. The round samples include at least state metadata and corresponding contribution labels. The contribution labels are obtained by the global model performance gain index caused by the participation of the vehicle client in training in the corresponding round. The performance gain index includes the decrease in the validation set prediction error and / or the decrease in the validation set loss.

[0069] In this embodiment, when the central server determines the set of vehicle clients participating in this round of training and allocates resource quotas based on the value assessment score, it includes: obtaining the resource cost estimate of each vehicle client; calculating the scheduling priority based on the value assessment score and the resource cost estimate; sorting by scheduling priority and selecting the top few vehicle clients to form a set under the constraints of communication resource budget and computing resource budget, and determining the communication resource quota and computing resource quota for each vehicle client in the set.

[0070] Scheduling priority is determined by both the value assessment score and the resource cost estimate. The central server performs a uniform scaling transformation on the value assessment score and the resource cost estimate before calculating the scheduling priority. When two or more vehicle clients have the same or indistinguishable scheduling priorities, the central server sorts them according to the order of the most recent successful parameter update upload time and prioritizes the vehicle client with the earlier most recent successful parameter update upload time to enter the set of vehicle clients participating in the current round of training.

[0071] In one implementation, during the calculation of scheduling priorities and the selection and allocation of quotas under resource budget constraints, the value assessment score and resource cost estimate obtained by the central server in the same round are unified to a comparable scale. Then, a monotonic scheduling priority is constructed, and set selection and quota allocation are completed under the constraints of communication resource budget and computing resource budget. Specifically:

[0072] When the The value assessment score for each vehicle client is: At that time, in the current round of candidate client set Normalization is performed internally:

[0073] ,

[0074] in, Indicates the first Normalized value assessment score for each vehicle client; Represents a set The minimum value of the intrinsic value assessment score; Represents a set The maximum value of the intrinsic value assessment score; This indicates a positive number smoothing term to prevent the denominator from being zero.

[0075] When resource overhead estimation includes uplink transmission latency estimation and local training latency estimation, they can be denoted as follows: and :

[0076] ,

[0077] in, Indicates the first Uplink transmission latency estimate for each vehicle client; Indicates the first Number of parameters to be uploaded by each vehicle client; Indicates the first Available uplink speed for each vehicle client; Indicates the first Local training latency estimation for each vehicle client; Indicates the maximum number of local training steps associated with computing resource quotas; Indicates the first The computing power metrics of each vehicle client.

[0078] When it is necessary to use overhead of different dimensions for the same priority function, it can be done in a set. Normalization is performed internally:

[0079] ,

[0080] in, Indicates the first Normalized communication overhead for each vehicle client; Represents a set Inside The minimum value; Represents a set Inside The maximum value; Indicates the first Normalized computational overhead for each vehicle client; Represents a set Inside The minimum value; Represents a set Inside The maximum value.

[0081] When considering compensation for clients that have not been selected for a long time to improve coverage, the number of rounds in which each vehicle client was not selected can be maintained. And update according to the scheduling results:

[0082] ,

[0083] in, Indicates the first The number of times each vehicle's client has been on the last selected wheel; Indicates the first Whether a vehicle client is selected in this round, the value can be 0 or 1.

[0084] Based on this, the following compensation options are acceptable:

[0085] ,

[0086] in, Indicates the first The compensation value for each vehicle client; Indicates the first Normalized compensation value for each vehicle client; Represents a set Inside The minimum value; Represents a set Inside The maximum value.

[0087] When value and overhead need to be combined into a monotonic scheduling priority, the following can be constructed:

[0088] ,

[0089] in, Indicates the first Scheduling priority for each vehicle client; This represents the communication overhead penalty coefficient; This indicates the calculation of the cost penalty coefficient; This represents the compensation weighting coefficient.

[0090] When selecting the set of vehicle clients to participate in the training round under the constraints of communication resource budget and computing resource budget, the selection set can be written as a utility maximization under two budget constraints:

[0091] ,

[0092] in, This indicates the current round of communication resource budget; This indicates the current round of computing resource budget; This represents the set of client indexes for candidate vehicles in this round.

[0093] When an feasible solution strategy is required, a greedy approximation based on normalized resource consumption can be used. To avoid Negative values ​​cause instability in the sorting, so truncation can be performed first:

[0094] ,

[0095] in, Indicates the first Non-negative priority for each vehicle client; Indicates the first Budget normalization priority for each vehicle client. Central server according to... Traverse in descending order and add the vehicle clients to the set one by one. And stop adding before any budget constraint is triggered, so that Both budget constraints mentioned above must be met.

[0096] When for a set When allocating communication and computing resource quotas to in-vehicle clients, the quotas can be combined with... Monotonic correlation, corresponding to forms such as the maximum uplink transmission bytes and the maximum number of local training rounds. Taking byte quotas and training step quotas as an example:

[0097] ,

[0098] in, Indicates the first Quota weight for each vehicle client; Indicates the first Communication resource quotas for each vehicle client; Indicates the lower limit of communication resource quota; This indicates the upper limit of communication resource quota.

[0099] ,

[0100] in, Indicates the first Computing resource quotas for each vehicle client; Indicates the lower limit of computing resource quota; This indicates the upper limit of the computational resource quota.

[0101] When there is a budget surplus after the quota is trimmed to the upper and lower limits, the remaining budget can be distributed among vehicle clients that have not reached the upper limit. The allocation is redistributed to keep the total quota in line with the budget and to give higher quotas to vehicle clients with higher scores and lower costs.

[0102] Specifically, this scheduling process unifies value assessment scores with estimated communication and computation costs on the same scale to calculate priorities, making priorities numerically comparable and maintaining a monotonic relationship with budget consumption. A penalty term is introduced into the priorities to account for communication and computation costs, preventing high-value but resource-intensive vehicle clients from monopolizing resources during budget constraints, thus reducing latency fluctuations in single-round training. A compensation term, using the number of rounds not selected as input, gives vehicle clients that have not participated for a long time a higher probability of being selected in subsequent rounds, improving data coverage and reducing long-term bias in the training sample distribution. Set selection aims to maximize utility under two types of budget constraints and is implemented using a greedy strategy with normalized budget consumption, allowing selection results to be quickly obtained on the server side. Quota allocation is monotonically correlated with priority weights and distributes the budget according to weights under upper and lower bound constraints, ensuring that high-priority vehicles receive higher communication and computation quotas while guaranteeing that the total quota does not exceed the budget.

[0103] In this embodiment, the communication resource quota includes the maximum number of uplink transmission bytes and / or the maximum number of uploads and / or the number of uplink time slots, and the calculated resource quota includes the upper limit of the number of local training rounds and / or the upper limit of the number of local training batches and / or the upper limit of local training duration.

[0104] In this embodiment, the resource overhead estimation includes at least uplink transmission latency estimation and local training latency estimation. The uplink transmission latency estimation is determined by the available uplink rate of the vehicle client and the amount of parameters to be uploaded. The amount of parameters to be uploaded is the amount of model parameter update data generated by the vehicle client after this round of local training and to be uploaded. The local training latency estimation is determined by the vehicle client's computing power index and computing resource quota. The computing power index is a characterization index of the vehicle client's ability to complete the number of local training steps within a preset time.

[0105] The available uplink rate is measured and reported by the vehicle client when reporting status metadata. The number of parameters to be uploaded is determined by the global model parameter scale and the encoding method of this round of upload. The computing power index is generated and reported by the processor capability information of the vehicle client and the training throughput statistics within the most recent preset time window. When the available uplink rate or computing power index reported by the vehicle client is unavailable, the central server uses the uplink transmission latency and local training latency obtained from the historical round statistics of the vehicle client as the resource overhead estimate.

[0106] Example 2:

[0107] Based on Example 1, when the central server aggregates model parameter updates, it performs weighted aggregation of the parameter updates according to the value assessment score corresponding to the vehicle client. The weight of the weighted aggregation is determined by the value assessment score and normalized. Furthermore, the central server does not receive parameter updates from vehicle clients that have not been scheduled.

[0108] This embodiment also provides a computer device, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize a data sharing and exchange method based on deep learning as proposed in the above embodiment.

[0109] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0110] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements a data sharing and exchange method based on deep learning as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0112] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this application and form different embodiments. For example, all the embodiments above can be used in any combination. The information disclosed in this background section is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that such information constitutes prior art known to those skilled in the art.

Claims

1. A data sharing and exchange method based on deep learning, applied to a federated learning system comprising a central server and multiple vehicle clients, characterized in that, include: Step S1: The central server obtains the status metadata reported by each vehicle client, inputs the status metadata into the pre-trained client data value dynamic evaluation network, and outputs the value evaluation score corresponding to each vehicle client. Step S2: Under the constraints of the given communication resource budget and computing resource budget, the central server determines the set of vehicle clients participating in this round of training based on the value assessment score, and allocates resource quotas to each vehicle client in the set. Step S3: Under the resource quota constraints, the vehicle clients within the set perform local model training and upload model parameter updates to the central server. Step S4: The central server aggregates the model parameter updates to update the global model; The status metadata includes at least: vehicle dynamic spatiotemporal information, real-time traffic event information associated with the dynamic spatiotemporal information, and vehicle equipment status quality information; wherein, the vehicle dynamic spatiotemporal information includes the position sequence and / or speed sequence within the most recent preset time window, the real-time traffic event information includes the event type, the road network element identifier corresponding to the event location, and the event timestamp, and the equipment status quality information includes positioning accuracy indicators, sensor effectiveness indicators, and / or data missing rate indicators; The client data value dynamic evaluation network includes at least: a graph neural network layer for encoding the traffic network topology map to output graph encoding results, a time-series coding layer for encoding the location sequence and / or speed sequence to output time-series coding results, an attention fusion layer for fusing the graph encoding results, the real-time traffic event information and the time-series coding results, and a fully connected layer for outputting the value evaluation score based on the fusion results. The pre-training of the client data value dynamic evaluation network is completed based on historical federated learning round samples. The round samples include at least state metadata and corresponding contribution labels. The contribution labels are obtained from the global model performance gain index caused by the participation of the vehicle client in the training in the corresponding round. The performance gain index includes the reduction in validation set prediction error and / or the reduction in validation set loss.

2. The data sharing and exchange method based on deep learning according to claim 1, characterized in that, When the central server determines the set of vehicle clients participating in this round of training and allocates resource quotas based on the value assessment score, it includes: obtaining the resource cost estimate of each vehicle client; calculating the scheduling priority based on the value assessment score and the resource cost estimate; sorting by scheduling priority and selecting the top few vehicle clients to form the set under the constraints of the communication resource budget and the computing resource budget, and determining the communication resource quota and computing resource quota for each vehicle client in the set.

3. The data sharing and exchange method based on deep learning according to claim 2, characterized in that, The communication resource quota includes the maximum number of uplink transmission bytes and / or the maximum number of uploads and / or the number of uplink time slots, and the computing resource quota includes the upper limit of the number of local training rounds and / or the upper limit of the number of local training batches and / or the upper limit of the local training duration.

4. The data sharing and exchange method based on deep learning according to claim 3, characterized in that, The resource overhead estimation includes at least uplink transmission latency estimation and local training latency estimation. The uplink transmission latency estimation is determined by the available uplink rate of the vehicle client and the amount of parameters to be uploaded. The amount of parameters to be uploaded is the amount of model parameter update data generated by the vehicle client after this round of local training and to be uploaded. The local training latency estimation is determined by the vehicle client's computing power index and the computing resource quota. The computing power index is a representation of the vehicle client's ability to complete the number of local training steps within a preset time.

5. The data sharing and exchange method based on deep learning according to claim 1, characterized in that, When the central server aggregates the model parameter updates, it performs weighted aggregation of the parameter updates according to the value assessment score corresponding to the vehicle client. The weight of the weighted aggregation is determined by the value assessment score and normalized. The central server does not receive parameter updates from vehicle clients that have not been scheduled.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the data sharing and exchange method based on deep learning as described in any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the data sharing and exchange method based on deep learning as described in any one of claims 1 to 5.

Images