Streaming time sequence model continuous learning method, device and equipment based on orthogonal rotation

Abstract

The invention relates to a streaming time sequence model continuous learning method, device and equipment based on orthogonal rotation. The method comprises the following steps: locking deep parameters of a pre-training time sequence model, sampling a sub-data set from each time window of historical traffic data, and inputting shallow-layer lightweight gradient updating for multiple times according to time; extracting and splicing a key matrix after each update, projecting the key matrix to a shallow weight matrix row space, and decomposing to obtain an input orthogonal matrix; constructing a covariance matrix, decomposing the covariance matrix to obtain an output orthogonal matrix, and updating shallow weight iteration to a condition to obtain a congestion prediction model; and inputting real-time traffic data to obtain a future road condition result. The method does not damage the original feature space, realizes continuous learning with high efficiency and low consumption, and is accurate in model prediction.

Description

TECHNICAL FIELD

[0001] The present application relates to the cross-technology field of streaming time series model continuous learning and road traffic flow analysis and congestion prediction, in particular to a streaming time series model continuous learning method based on orthogonal rotation, device and equipment. BACKGROUND

[0002] With the acceleration of urbanization and the surge of motor vehicles, urban traffic congestion has become a core pain point. Accurate traffic flow (congestion state) prediction is the key to intelligent transportation system (ITS) dynamic control, congestion warning and path optimization. However, urban traffic flow data has the characteristics of "streaming dynamics, variable distribution, and multi-source coupling": data is generated in real time by roadside devices and is easily disturbed by factors such as morning and evening rush hours, construction, accidents, weather, and policies, resulting in data distribution and prediction targets drifting over time, requiring high online adaptation capabilities for models.

[0003] Existing traffic flow prediction technologies have obvious limitations: early statistical models (such as ARIMA) and deep learning models (such as LSTM) have weak generalization ability and need to be retrained to face cross-road section distribution differences, which cannot adapt to urban-level streaming scenarios. In recent years, time series basic models (TSFMs such as Timer and PatchTST) have universal time series representation capabilities through pre-training, but have large parameter sizes (tens of millions to hundreds of millions), which will face "catastrophic forgetting" if fine-tuned to adapt to flow drift, i.e. new flow patterns cover historical congestion laws and "resource bottlenecks", i.e. high-frequency gradient updates are difficult to adapt to edge devices.

[0004] While model editing methods (such as ROME and AlphaEdit) attempt to update parameters locally, they still have defects when applied to traffic scenarios: first, the feature space drifts, and the road section association of traffic flow (such as congestion conduction) is destroyed by cumulative parameter disturbance; second, there is a lack of cross-time consistency, and the common laws of historical and current traffic (such as the early morning peak period) are not integrated; third, the storage overhead is large, and it is difficult to deploy accumulated historical data or editing matrices for a long time. SUMMARY

[0005] Therefore, it is necessary to provide a streaming time series model continuous learning method based on orthogonal rotation that can stably adapt to traffic flow with low consumption and anti-forgetting.

[0006] A streaming time series model continuous learning method based on orthogonal rotation, the method comprising: obtaining a pre-trained time series basic model comprising a shallow layer and a deep layer network, and locking the parameters of the deep layer network; obtain historical training data, which includes road traffic volume historical data segments divided into multiple time windows, randomly sample each road traffic volume historical data segment according to a preset proportion to obtain multiple sub-data sets; input each of the sub-data sets to the shallow network in sequence according to time for multiple times of light gradient update, extract an input key matrix and an output value matrix of the shallow network after each time of light gradient update, and splice the input key matrix with an input key matrix after last time of light gradient update to obtain a spliced key matrix; project the spliced key matrix into a row space of a weight matrix of the shallow network based on a row space basis of the current weight matrix of the shallow network, and perform singular value decomposition on a projection result to obtain an input space alignment orthogonal matrix; construct a covariance matrix according to an intermediate output matrix calculated according to the input space alignment orthogonal matrix and the output value matrix, and perform singular value decomposition on the covariance matrix to construct an output space correction orthogonal matrix meeting a preset condition according to a decomposition result; update the current weight matrix of the shallow network according to the output space correction orthogonal matrix, and perform next time of light gradient update by using the shallow network after weight matrix update, until a preset condition is met, and then obtain a congestion prediction model adapting to a current traffic flow distribution; input a real-time road traffic volume data segment into the congestion prediction model to obtain a prediction result of a road congestion situation at a future target time point.

[0007] In one of the embodiments, the time series base model adopts a Transformer architecture. The shallow network is the first to second layers of the Transformer architecture. The deep network is the third layer and all layers after the third layer of the Transformer architecture.

[0008] In one of the embodiments, the input space alignment orthogonal matrix is a right singular vector matrix in a decomposition result obtained by performing singular value decomposition on the projection result.

[0009] In one of the embodiments, constructing a covariance matrix according to an intermediate output matrix calculated according to the input space alignment orthogonal matrix and the output value matrix includes: constructing a first intermediate output matrix and a second intermediate output matrix according to a current input key matrix and an input key matrix after last time of light gradient update based on the input space alignment orthogonal matrix and the weight matrix respectively; constructing the covariance matrix according to the first intermediate output matrix, the second intermediate output matrix, a current output value matrix and an output value matrix after last time of light gradient update.

[0010] In one embodiment, constructing the output space correction orthogonal matrix satisfying the preset condition based on the singular value decomposition result of the covariance matrix comprises: The singular value decomposition result of the covariance matrix comprises a left singular vector matrix and a right singular vector matrix; The output space correction orthogonal matrix is constructed by multiplying the transpose of the left singular vector matrix and the right singular vector matrix; The preset condition comprises that the output space correction orthogonal matrix satisfies the orthogonal constraint that the transpose of the output space correction orthogonal matrix multiplied by the output space correction orthogonal matrix equals a unit matrix; And the output space correction orthogonal matrix needs to minimize the following formula: ; In the above formula, denotes the output space correction orthogonal matrix, and denote the first intermediate output matrix and the second intermediate output matrix, and denote the current output value matrix and the output value matrix after the last light gradient update.

[0011] In one embodiment, when updating the current weight matrix of the shallow network according to the output space correction orthogonal matrix, the following formula is used: ; In the above formula, and denote the updated weight matrix and the current weight matrix, denotes the output space correction orthogonal matrix, denotes the transpose of the input space alignment orthogonal matrix.

[0012] In one embodiment, before updating the current weight matrix of the shallow network: Perform truncated singular value decomposition on the current weight matrix to construct a low-rank approximation matrix based on components corresponding to the first preset number of maximum singular values; The low-rank approximation matrix is used in projecting the concatenation key matrix into the row space of the weight matrix and in singular value decomposition of the covariance matrix.

[0013] The application also provides an orthogonal rotation-based streaming time sequence model continuous learning device, which comprises: A pre-trained time sequence base model acquisition module is configured to acquire a pre-trained time sequence base model comprising a shallow network and a deep network, and lock the parameters of the deep network; The sub-data set acquisition module is configured to acquire historical training data, which includes road traffic volume historical data segments divided into multiple time windows, and randomly sample each road traffic volume historical data segment according to a preset proportion to obtain multiple corresponding sub-data sets; The spliced key matrix obtaining module is configured to sequentially input each of the sub-data sets to the shallow network for multiple times of lightweight gradient update, extract an input key matrix and an output value matrix of the shallow network after each time of lightweight gradient update, splice the input key matrix with an input key matrix after the last time of lightweight gradient update to obtain a spliced key matrix; The input space alignment orthogonal matrix obtaining module is configured to project the spliced key matrix into a row space of a weight matrix of the shallow network based on a row space basis of the current weight matrix of the shallow network, and perform singular value decomposition on a projection result to obtain an input space alignment orthogonal matrix; The output space correction orthogonal matrix obtaining module is configured to construct a covariance matrix according to an intermediate output matrix calculated based on the input space alignment orthogonal matrix and the output value matrix, and perform singular value decomposition on the covariance matrix to construct an output space correction orthogonal matrix satisfying a preset condition; The congestion prediction model obtaining module is configured to update the current weight matrix of the shallow network according to the output space correction orthogonal matrix, and perform next time of lightweight gradient update on the shallow network after the weight matrix is updated, until a preset condition is satisfied, and then obtain a congestion prediction model adapted to a current traffic flow distribution. The road congestion condition prediction module is configured to input a real-time road traffic volume data segment into the congestion prediction model to obtain a road congestion condition prediction result at a future target time point.

[0014] A computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps in the above-mentioned orthogonal rotation-based streaming time series model continuous learning method when executing the computer program.

[0015] A computer readable storage medium stores a computer program, and the computer program implements the steps in the orthogonal rotation-based streaming time series model continuous learning method when executed by a processor.

[0016] The method, device and equipment for continuous learning of the flow-based time sequence model based on orthogonal rotation can lock the parameters of the deep network in the pre-trained time sequence base model including the shallow network and the deep network, randomly sample the road traffic history data segments of each time window from the historical training data according to a preset proportion, obtain a plurality of sub-data sets, input each sub-data set to the shallow network in sequence according to time for multiple times of lightweight gradient update, extract the input key matrix and the output value matrix of the updated shallow network after each time of lightweight gradient update, splice the input key matrix with the input key matrix after the last time of lightweight gradient update to obtain a spliced key matrix, project the spliced key matrix into the row space of the weight matrix based on the row space basis of the current weight matrix of the shallow network, perform singular value decomposition on the projection result to obtain an input space alignment orthogonal matrix, construct a covariance matrix according to the intermediate output matrix calculated based on the input space alignment orthogonal matrix and the output value matrix, perform singular value decomposition on the covariance matrix, construct an output space correction orthogonal matrix meeting a preset condition according to the decomposition result, update the current weight matrix of the shallow network according to the output space correction orthogonal matrix, perform the next time of lightweight gradient update by using the shallow network with the updated weight matrix, and until the preset condition is met, a congestion prediction model adapted to the current traffic flow distribution is obtained. The real-time road traffic data segment is input into the congestion prediction model to obtain the prediction result of the road congestion at the future target time point. By using the method, the time sequence base model can be continuously learned in the flow data scene efficiently, stably and with low cost without destroying the feature space structure of the original model, so that the neural network for predicting the road condition at the future time point from the real-time road traffic data segment is trained efficiently and with low cost, and the road condition prediction result is accurately obtained. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A flowchart of the method for continuous learning of the flow-based time sequence model based on orthogonal rotation in one embodiment; Figure 2 A schematic diagram of the attention weight distribution and orthogonal transformation effect of the TSFMs in one embodiment; Figure 3 A schematic diagram of the data processing flow of the method for continuous learning of the flow-based time sequence model based on orthogonal rotation in one embodiment; Figure 4 A structural block diagram of the device for continuous learning of the flow-based time sequence model based on orthogonal rotation in one embodiment; Figure 5 An internal structure diagram of the computer device in one embodiment. DETAILED DESCRIPTION

[0018] In order to make the purposes, technical solutions and advantages of the present application clearer, further detailed description will be made to the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0019] In the present application, as shown in Figure 1 An orthogonal rotation-based streaming time sequence model continuous learning method is provided, which specifically comprises the following steps: Step S100, a pre-trained time sequence base model comprising a shallow network and a deep network is obtained, and the parameters of the deep network are locked.

[0020] Step S110, historical training data is obtained, which comprises road traffic volume historical data segments divided into multiple time windows, and a plurality of sub-data sets are obtained by randomly sampling the road traffic volume historical data segments according to a preset proportion.

[0021] Step S120, each sub-data set is sequentially input to the shallow network for multiple times of lightweight gradient update, and the input key matrix and the output value matrix of the current updated shallow network are extracted after each time of lightweight gradient update, and the input key matrix is spliced with the input key matrix after the last time of lightweight gradient update to obtain a spliced key matrix.

[0022] Step S130, based on the row space basis of the current weight matrix of the shallow network, the spliced key matrix is projected into the row space of the weight matrix, and the input space alignment orthogonal matrix is obtained by singular value decomposition of the projection result.

[0023] Step S140, the covariance matrix is constructed according to the intermediate output matrix calculated from the input space alignment orthogonal matrix and the output value matrix, and the output space correction orthogonal matrix satisfying the preset condition is constructed according to the decomposition result of the singular value decomposition of the covariance matrix.

[0024] Step S150, the current weight matrix of the shallow network is updated according to the output space correction orthogonal matrix, and the next time of lightweight gradient update is performed by using the shallow network with the updated weight matrix, until the preset condition is satisfied, and then the congestion prediction model adapted to the current traffic flow distribution is obtained.

[0025] Step S160, a real-time road traffic volume data segment is input into the congestion prediction model to obtain a prediction result of the road congestion at a future target time point.

[0026] In this application, addressing the challenges of dynamic evolution of streaming traffic data (such as real-time road segment speed and vehicle density), cross-domain distribution drift (such as sudden changes in traffic flow during rainy days / holidays), and limited roadside equipment resources in road traffic flow analysis and congestion prediction scenarios, an orthogonal rotation transformation mechanism is introduced. This mechanism strictly maintains the geometric structure of the original feature space during continuous learning, effectively preserving historical traffic knowledge (such as the congestion transmission patterns during morning rush hour and the regular traffic efficiency of road segments), avoiding catastrophic forgetting, and ensuring that the model does not lose key historical prediction experience due to updates of new traffic flow data. Furthermore, by updating only shallow network parameters and combining targeted sampling of road traffic flow data (such as prioritizing the retention of key traffic flow samples during peak hours) and a truncated singular value decomposition strategy, computational complexity and storage overhead are significantly reduced. This allows for adaptation to low-resource devices such as roadside terminals, achieving efficient real-time processing of traffic flow data and dynamic model adaptation. This method is highly versatile and can be seamlessly applied to various Transformer-based time series models (such as Timer, Moirai, PatchTST, etc.). It can stably learn and accurately output congestion prediction results in complex flow scenarios such as single-segment traffic flow evolution (such as weekday traffic flow pattern changes) and cross-regional traffic flow distribution drift (such as traffic flow switching between commercial districts and residential areas).

[0027] Furthermore, the framework relies on the isometric invariance of orthogonal transformations and the closed-form optimal solution of the Orthogonal Procrustes problem to provide a solid theoretical guarantee for the consistency of road traffic flow feature extraction and the reliability of congestion prediction results, thereby further improving the robustness of the model in complex traffic scenarios.

[0028] In step S100, the temporal base model adopts the Transformer architecture, where the shallow network consists of the first 1 to 2 layers of the Transformer architecture, and the deep network consists of the 3rd layer and all subsequent layers of the Transformer architecture. During training, only the feedforward network (FFN) weight matrix of the shallow network (such as the first two layers of the Transformer) is exposed. For future updates.

[0029] In this embodiment, the pre-trained temporal base model is a Timer model that has been pre-trained on a large-scale general temporal corpus (such as UTSD-12G), such as... Figure 2As shown, attention weights are significantly concentrated in the shallow layers, indicating that they are responsible for extracting basic temporal patterns (such as periodicity, trends, and abrupt changes); while the deep networks mainly perform feature combination and semantic abstraction, exhibiting greater versatility. Freezing the deep layer parameters effectively preserves the cross-domain knowledge acquired by the model during pre-training, avoiding semantic drift caused by frequent updates. Simultaneously, limiting the update scope to the shallow layers significantly reduces the degrees of freedom and computational complexity of parameter updates, laying the foundation for subsequent efficient orthogonal transformations. This design achieves a crucial balance between "adaptability" and "stability."

[0030] In step S110, the historical training data specifically refers to road traffic flow observation data accumulated over multiple time periods, covering different traffic scenarios (such as weekday morning rush hour, holiday off-peak hours, and commuting periods during rainy days). It is divided into multiple non-overlapping time windows in chronological order, with each time window corresponding to one segment of historical road traffic flow data. The core components of each data segment are consistent with the characteristics of streaming data, namely, they include... A sample, denoted as ,in This is the "current road traffic flow input sequence" for the i-th sample within this time window (L is the sequence length, corresponding to the time step within the window; e.g., L=10 represents 10 five-minute observation points; d is the feature dimension, covering d-type core traffic flow indicators such as number of lanes, average speed, vehicle density, and road occupancy). Simultaneously, each road traffic flow historical data segment also has a corresponding road traffic prediction target for the next H steps, represented as... As a truth label, H is the prediction step size. For example, H=3 represents the next 15 minutes. The prediction targets include indicators directly related to congestion prediction, such as vehicle density, congestion level, and travel time in the next H steps.

[0031] Considering the sample size of historical traffic flow data segments for each road Typically, the data is quite large (e.g., a single morning peak window can contain thousands of traffic flow observation records). Updating the model on the entire sample would lead to insufficient resources for edge devices such as roadside terminals due to excessive computational load (e.g., exceeding memory limits, excessively long gradient update times). Therefore, this application adopts a random sampling strategy consistent with streaming data for each road traffic flow historical data segment: according to a preset proportion. For example, α = 0.01, 0.05, or 0.1. The specific value can be dynamically adjusted according to the computing power of the equipment to extract samples from each data segment and form corresponding subsets. The number of samples in its subset is .

[0032] Specifically, based on statistical learning theory, under the Lipschitz continuity assumption (i.e., traffic flow characteristics change smoothly over time without extreme jumps), the upper bound of the estimation error of each subset of data points for the mean (e.g., average vehicle speed, average vehicle density) and variance (e.g., traffic flow fluctuation amplitude) of the original data segment is: For example, when and At the same time, the upper bound of the error can be controlled within a small range, ensuring that the subset of data can accurately reflect the traffic flow distribution characteristics of the original road traffic volume historical data segment (such as the peak traffic flow period during the morning rush hour and the congestion transmission pattern of road segments), while reducing the computational burden to full processing. This provides an efficient data foundation for subsequent steps such as lightweight gradient updates and orthogonal matrix calculations in shallow networks.

[0033] In step S120, the subset dataset is used The shallow network is then updated multiple times using lightweight gradients based on the corresponding ground truth labels. This process can be represented as follows: ; In the above formula, is the learning rate. This fine-tuning aims to initially adapt the model to the current data distribution, thereby extracting more accurate key-value pairs.

[0034] Furthermore, forward propagation again Record the input key matrix of the shallow FFN. With the output value matrix ,in At the same time, the system caches the previous time step. This is used to build consistency constraints later and prevent them from being forgotten.

[0035] In step 130, the input space alignment orthogonal matrix is ​​calculated. Its core motivation is to ensure that both new and old input features are mapped to the original weight matrix. Effective row space (i.e.) Within this range, we avoid introducing new directions that are unrelated to historical knowledge, thereby maintaining the consistency of the input feature space.

[0036] In this embodiment, the current weight matrix is ​​first... The process of performing Singular Value Decomposition (SVD) is represented as follows: ; In the above formula, Let be the left and right singular vector matrices. It is a singular value diagonal matrix. .matrix The column vectors constitute A set of orthonormal bases.

[0037] Furthermore, the input key matrix Compared to the input key matrix after the last lightweight gradient update Perform horizontal concatenation to obtain the concatenation key matrix. Through orthogonal projection operator Project it to The process is represented as follows: ; The above operation forces all input features to be located in Within its scope. Subsequently, on Performing Singular Value Decomposition (SVD) is represented as follows: ; In the above formula, It is an orthogonal matrix, that is, the right singular vector matrix of the singular value decomposition result, and its column vectors span by . And align the input space with orthogonal matrices. The transformed input satisfy: ; In this embodiment, if the input key matrix is ​​detected Compared to the input key matrix after the last lightweight gradient update The included angle of the main directions exceeds the preset threshold. This triggers the "spatial realignment mechanism" on the right singular vector matrix. Dynamic expansion or incremental SVD can be used to enhance the model's robustness to sudden distribution shifts.

[0038] In step S140, the output space correction orthogonality matrix is ​​calculated. Its design motivation is to further correct the output direction, given that the inputs are already aligned, so that the updated model can accurately fit both the old and new target outputs, i.e., the current output value matrix. The output matrix after the last lightweight gradient update This achieves optimal alignment of the output space.

[0039] In this embodiment, the covariance matrix is ​​constructed based on the intermediate output matrix obtained by aligning the input space orthogonal matrix and the output value matrix. This includes: constructing a first intermediate output matrix and a second intermediate output matrix based on the current input key matrix and the input key matrix after the last lightweight gradient update, respectively, based on the input space orthogonal matrix and the weight matrix; and then constructing the covariance matrix based on the first intermediate output matrix, the second intermediate output matrix, the current output value matrix, and the output value matrix after the last lightweight gradient update.

[0040] Specifically, the first intermediate output matrix and the second intermediate output matrix are represented as follows: ; In the above formula, and These represent the first intermediate output matrix and the second intermediate output matrix, respectively. Current weight matrix, This represents the transpose of the input space-aligned orthogonal matrix.

[0041] Furthermore, the output space orthogonal calibration matrix, constructed from the singular value decomposition (SVD) results of the covariance matrix, satisfies the following preset conditions: the SVD results of the covariance matrix include a left singular vector matrix and a right singular vector matrix; the output space orthogonal calibration matrix is ​​constructed by multiplying the transposes of the left and right singular vector matrices. Simultaneously, the preset conditions for the obtained output space orthogonal calibration matrix include: the output space orthogonal calibration matrix satisfies the orthogonality constraint that its transpose multiplied by itself equals the identity matrix; and the output space orthogonal calibration matrix must minimize the following formula: ; In the above formula, This represents the output space correction orthogonal matrix. and These represent the first intermediate output matrix and the second intermediate output matrix, respectively. and These represent the current output matrix and the output matrix after the last lightweight gradient update, respectively.

[0042] Specifically, after obtaining the first intermediate output matrix and the second intermediate output matrix Later, due to Therefore The goal is to find orthogonal matrices. (satisfy ), so that: ; This problem is equivalent to the classic Orthogonal Procrustes problem, and its optimization objective is: ; Through the transformation of the trace identity, this problem is equivalent to maximizing The covariance matrix Defined as: ; right Execute SVD: ; The optimal solution is: ; The solution minimizes the Frobenius norm error between the intermediate output and the target output under orthogonal constraints, thus possessing theoretical optimality.

[0043] In this embodiment, a regularization term is also introduced during the calculation of the output space correction orthogonality matrix. In the Procrustes objective function, the output rotation magnitude is controlled to avoid semantic distortion caused by overcorrection. It can be learned hyperparameters or adaptively adjusted according to the degree of task drift.

[0044] In step S150, when updating the current weight matrix of the shallow network based on the output space correction orthogonal matrix, the following formula is used: ; In the above formula, and These represent the updated weight matrix and the current weight matrix, respectively. This represents the output space correction orthogonality matrix. This represents the transpose of the input space-aligned orthogonal matrix.

[0045] In this embodiment, the above-described update has three key properties: First, it is isometric in the feature space: for any input ,have Because orthogonal matrices preserve norm, this ensures that the transformed feature distribution is isomorphic to the original space, avoiding structural distortion. Secondly, it ensures invariance in representational power. and They have the same singular value spectrum, i.e. Model capacity is preserved. Finally, historical task compatibility is ensured by Procrustes optimization. This effectively alleviates catastrophic forgetting. (Updated version) It is written back to the model and used for inference at the next time step.

[0046] In this embodiment, to address the computational and storage bottlenecks caused by high-dimensional parameter matrices, truncated SVD is introduced for low-rank approximation. Before updating the current weight matrix of the shallow network: truncated singular value decomposition is performed on the current weight matrix, and a low-rank approximation matrix is ​​constructed based on the components corresponding to the largest singular values ​​of the previous preset number. The low-rank approximation matrix is ​​used when projecting the concatenation key matrix into the row space of the weight matrix and when performing singular value decomposition on the covariance matrix.

[0047] Specifically, for the original weight matrix Keep only the previous one The components corresponding to the largest singular values ​​( ),have to: ; In the above formula, According to the Eckart–Young–Mirsky theorem, It is all ranks In the matrix, the weight matrix The best Frobenius norm approximation. All SVD operations in subsequent steps S130 and S140 are based on Perform this process to make the orthogonal matrix The dimensionality is significantly reduced, and the storage overhead is reduced from Down to In actual deployment, It can be set to 64 or 128, achieving an order-of-magnitude efficiency improvement within an acceptable range of accuracy loss.

[0048] In this embodiment, after updating the weight matrix, the performance change of the model on historical tasks before and after the update is calculated using a validation subset. If the decline exceeds the tolerance threshold, the model is rolled back to the previous version of parameters and a recalibration process is triggered to ensure the stability of the continuous learning process.

[0049] like Figure 3 The diagram shows a data processing flow diagram of a streaming time series model continuous learning method based on orthogonal rotation.

[0050] In the aforementioned continuous learning method for streaming time-series models based on orthogonal rotation, by locking the deep parameters of the underlying time-series model and concatenating the current and previous input key matrices after each lightweight gradient update, it can both retain the historical global traffic knowledge stored in deep storage (such as the congestion transmission patterns of multiple road segments) and associate it with historical traffic flow features, effectively avoiding catastrophic forgetting in continuous learning. By sampling subsets of road traffic flow data from each time window according to a preset ratio, combined with the design of updating only shallow network parameters, the computational and storage overhead of full data processing is significantly reduced, adapting to the resource constraints of roadside edge devices. The data set is input sequentially over time, and the concatenation key matrix is ​​projected onto the row space of the weight matrix by aligning the orthogonal matrix in the input space. This aligns with dynamically shifting traffic flow characteristics (such as traffic flow changes from morning rush hour to off-peak hours, and from sunny days to rainy days), solving the problem of poor model adaptability caused by the distribution drift of streaming data. Then, the covariance matrix associated with the real congestion target (such as the future H-step traffic density) is constructed by correcting the orthogonal matrix in the output space, and the shallow weights are iteratively updated to ensure that the model output is close to the prediction target. Ultimately, this improves the accuracy of road congestion prediction and enables the model to learn efficiently and stably in complex streaming traffic scenarios.

[0051] It should be understood that, although Figure 1The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0052] In one embodiment, such as Figure 4 As shown, a continuous learning device for streaming time series models based on orthogonal rotation is provided, including: a pre-trained time series basic model acquisition module 200, a subset dataset acquisition module 210, a concatenation key matrix acquisition module 220, an input space aligned orthogonal matrix acquisition module 230, an output space corrected orthogonal matrix acquisition module 240, a congestion prediction model acquisition module 250, and a road congestion prediction module 260, wherein: The pre-trained temporal base model acquisition module 200 is used to acquire pre-trained temporal base models including shallow and deep networks, and to lock the parameters of deep networks. The subset acquisition module 210 is used to acquire historical training data, which includes road traffic flow historical data segments divided into multiple time windows. Random sampling is performed from each road traffic flow historical data segment according to a preset ratio to obtain multiple corresponding subsets. The concatenation key matrix module 220 is used to input each of the aforementioned subset datasets into the shallow network in chronological order for multiple lightweight gradient updates. After each lightweight gradient update, the input key matrix and output value matrix of the shallow network after the current update are extracted, and the input key matrix is ​​concatenated with the input key matrix after the previous lightweight gradient update to obtain the concatenation key matrix. The input space aligned orthogonal matrix obtaining module 230 is used to project the splicing key matrix into the row space of the weight matrix based on the row space basis of the current weight matrix of the shallow network, and perform singular value decomposition on the projection result to obtain the input space aligned orthogonal matrix; The output space correction orthogonal matrix obtaining module 240 is used to construct a covariance matrix based on the intermediate output matrix calculated from the input space alignment orthogonal matrix and the output value matrix, and then perform singular value decomposition on the covariance matrix to construct an output space correction orthogonal matrix that satisfies preset conditions based on the decomposition results. The congestion prediction model module 250 is used to update the current weight matrix of the shallow network according to the output space correction orthogonal matrix. The shallow network updated with the weight matrix is ​​then used for the next lightweight gradient update until the preset conditions are met, thus obtaining a congestion prediction model that adapts to the current traffic flow distribution. The road congestion prediction module 260 is used to input real-time road traffic flow data segments into the congestion prediction model to obtain the road congestion prediction results for future target time points.

[0053] Specific limitations regarding the continuous learning device for streaming time series models based on orthogonal rotation can be found in the limitations of the continuous learning method for streaming time series models based on orthogonal rotation mentioned above, and will not be repeated here. Each module in the aforementioned continuous learning device for streaming time series models based on orthogonal rotation can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0054] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a continuous learning method for a streaming time-series model based on orthogonal rotation. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0055] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0056] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps: Obtain a pre-trained temporal base model including shallow and deep networks, and lock the parameters of the deep network; Historical training data is acquired, which includes historical road traffic flow data segments divided into multiple time windows. Random sampling is performed from each historical road traffic flow data segment according to a preset ratio to obtain multiple corresponding subset datasets. Each of the aforementioned subsets of data is sequentially input into the shallow network in chronological order for multiple lightweight gradient updates. After each lightweight gradient update, the input key matrix and output value matrix of the shallow network after the current update are extracted, and the input key matrix is ​​concatenated with the input key matrix after the previous lightweight gradient update to obtain the concatenated key matrix. Based on the row space basis of the current weight matrix of the shallow network, the splicing key matrix is ​​projected into the row space of the weight matrix, and the projection result is subjected to singular value decomposition to obtain the input space aligned orthogonal matrix. The intermediate output matrix is ​​calculated based on the input space alignment orthogonal matrix, and a covariance matrix is ​​constructed based on the output value matrix. Then, singular value decomposition is performed on the covariance matrix, and an output space correction orthogonal matrix that meets the preset conditions is constructed based on the decomposition results. The current weight matrix of the shallow network is updated based on the output space correction orthogonal matrix. The shallow network updated with the weight matrix is ​​then used for the next lightweight gradient update until the preset conditions are met, thus obtaining a congestion prediction model that adapts to the current traffic flow distribution. The real-time road traffic flow data segment is input into the congestion prediction model to obtain the prediction result of road congestion at the future target time point.

[0057] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor: Obtain a pre-trained temporal base model including shallow and deep networks, and lock the parameters of the deep network; Historical training data is acquired, which includes historical road traffic flow data segments divided into multiple time windows. Random sampling is performed from each historical road traffic flow data segment according to a preset ratio to obtain multiple corresponding subset datasets. Each of the aforementioned subsets of data is sequentially input into the shallow network in chronological order for multiple lightweight gradient updates. After each lightweight gradient update, the input key matrix and output value matrix of the shallow network after the current update are extracted, and the input key matrix is ​​concatenated with the input key matrix after the previous lightweight gradient update to obtain the concatenated key matrix. Based on the row space basis of the current weight matrix of the shallow network, the splicing key matrix is ​​projected into the row space of the weight matrix, and the projection result is subjected to singular value decomposition to obtain the input space aligned orthogonal matrix. The intermediate output matrix is ​​calculated based on the input space alignment orthogonal matrix, and a covariance matrix is ​​constructed based on the output value matrix. Then, singular value decomposition is performed on the covariance matrix, and an output space correction orthogonal matrix that meets the preset conditions is constructed based on the decomposition results. The current weight matrix of the shallow network is updated based on the output space correction orthogonal matrix. The shallow network updated with the weight matrix is ​​then used for the next lightweight gradient update until the preset conditions are met, thus obtaining a congestion prediction model that adapts to the current traffic flow distribution. The real-time road traffic flow data segment is input into the congestion prediction model to obtain the prediction result of road congestion at the future target time point.

[0058] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0059] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0060] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A continuous learning method for streaming time series models based on orthogonal rotation, characterized in that, The method comprises: acquiring a pre-training time series base model comprising a shallow network and a deep network, and locking parameters of the deep network; acquiring historical training data comprising road traffic volume historical data segments divided into multiple time windows, and randomly sampling the road traffic volume historical data segments according to a preset proportion to obtain multiple sub-data sets; inputting each of the sub-data sets into the shallow network in sequence according to time for multiple times of light gradient updating, extracting an input key matrix and an output value matrix of the shallow network after each time of light gradient updating, and splicing the input key matrix with an input key matrix after last time of light gradient updating to obtain a spliced key matrix; projecting the spliced key matrix into a row space of a current weight matrix of the shallow network based on a row space basis of the current weight matrix, and performing singular value decomposition on a projection result to obtain an input space alignment orthogonal matrix; constructing a covariance matrix according to an intermediate output matrix calculated based on the input space alignment orthogonal matrix and the output value matrix, and performing singular value decomposition on the covariance matrix to construct an output space correction orthogonal matrix satisfying a preset condition according to a decomposition result; updating the current weight matrix of the shallow network according to the output space correction orthogonal matrix, and performing next time of light gradient updating by using the shallow network after weight matrix updating, until a preset condition is satisfied, to obtain a congestion prediction model adapted to a current traffic flow distribution; inputting a real-time road traffic volume data segment into the congestion prediction model to obtain a prediction result of a road congestion condition at a future target time point.

2. The orthogonal rotation-based recurrent temporal model continual learning method according to claim 1, characterized in that, The time series base model adopts a Transformer architecture; The shallow network is the first to second layers of the Transformer architecture; The deep network is the third layer and all layers thereafter of the Transformer architecture. 3.The orthogonal rotation-based recurrent temporal model continual learning method according to claim 2, characterized in that, The input space alignment orthogonal matrix is a right singular vector matrix in a decomposition result obtained by performing singular value decomposition on the projection result.

4. The orthogonal rotation-based recurrent temporal model continual learning method according to claim 3, characterized in that, Constructing a covariance matrix according to an intermediate output matrix calculated based on the input space alignment orthogonal matrix and the output value matrix comprises: constructing a first intermediate output matrix and a second intermediate output matrix according to a current input key matrix and an input key matrix after last time of light gradient updating based on the input space alignment orthogonal matrix and a weight matrix, respectively; constructing the covariance matrix according to the first intermediate output matrix, the second intermediate output matrix, a current output value matrix and an output value matrix after last time of light gradient updating.

5. The orthogonal rotation-based recurrent temporal model continual learning method according to claim 4, characterized in that, The singular value decomposition result of the covariance matrix comprises a left singular vector matrix and a right singular vector matrix; The output space correction orthogonal matrix is constructed by multiplying a transpose of the left singular vector matrix with the right singular vector matrix; The preset condition comprises that the output space correction orthogonal matrix satisfies an orthogonal constraint that a transpose of the output space correction orthogonal matrix multiplied by the output space correction orthogonal matrix is equal to a unit matrix; and the output space correction orthogonal matrix needs to minimize the following formula: ​ ​ ； In the above formula, denotes the output space correction orthogonal matrix, and denote the first intermediate output matrix, second intermediate output matrix, and denote the current output value matrix and the output value matrix after the last light gradient update, respectively.

6. The orthogonal rotation-based recurrent sequential model continual learning method according to claim 5, characterized in that, When updating the current weight matrix of the shallow network according to the output space correction orthogonal matrix, the following formula is adopted: ； In the above formulae, and respectively represent the updated weight matrix and the current weight matrix, represents the output space correction orthogonal matrix, represents the transpose of the input space alignment orthogonal matrix.

7. The orthogonal rotation-based recurrent temporal model continual learning method according to claim 5, characterized in that, Before updating the current weight matrix of the shallow network: Perform truncated singular value decomposition on the current weight matrix, and construct a low-rank approximation matrix according to components corresponding to the first preset number of maximum singular values; Both the projection of the spliced key matrix into the row space of the weight matrix and the singular value decomposition of the covariance matrix are based on the low-rank approximation matrix.

8. An orthogonal rotation-based stream-wise temporal model continuous learning device, characterized in that, The device comprises: A pre-trained time series base model acquisition module is configured to acquire a pre-trained time series base model comprising a shallow network and a deep network, and lock the parameters of the deep network; A sub-data set acquisition module is configured to acquire historical training data, which comprises road traffic volume historical data segments divided into multiple time windows, and obtain multiple sub-data sets by randomly sampling the road traffic volume historical data segments according to a preset proportion; A spliced key matrix acquisition module is configured to sequentially input each of the sub-data sets into the shallow network for multiple times of lightweight gradient update, extract an input key matrix and an output value matrix of the shallow network after each time of lightweight gradient update, and splice the input key matrix with an input key matrix after the last time of lightweight gradient update to obtain a spliced key matrix; An input space alignment orthogonal matrix acquisition module is configured to project the spliced key matrix into a row space of a weight matrix of the shallow network based on a row space basis of the current weight matrix, and perform singular value decomposition on a projection result to obtain an input space alignment orthogonal matrix; An output space correction orthogonal matrix acquisition module is configured to construct a covariance matrix based on an intermediate output matrix calculated according to the input space alignment orthogonal matrix and the output value matrix, and perform singular value decomposition on the covariance matrix to construct an output space correction orthogonal matrix satisfying a preset condition according to a decomposition result; A congestion prediction model acquisition module is configured to update the current weight matrix of the shallow network according to the output space correction orthogonal matrix, perform the next time of lightweight gradient update on the shallow network after the weight matrix is updated, and obtain a congestion prediction model adapted to a current traffic flow distribution until a preset condition is satisfied; A road congestion condition prediction module is configured to input a real-time road traffic volume data segment into the congestion prediction model to obtain a road congestion condition prediction result at a future target time point. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor implements the steps of the method of any one of claims 1 to 7 when executing the computer program. The processor implements the steps of the method of any one of claims 1 to 7 when executing the computer program.

Images