Intelligent linkage method and system of cloud desktop workstation and big data server

Abstract

The present application relates to the technical field of data processing and cloud computing, and particularly relates to an intelligent linkage method and system of a cloud desktop workstation and a big data server, the cloud desktop workstation continuously collects streaming data and generates data stream units with time stamps and modal labels, and pushes the data stream units to the big data server through asynchronous transmission, the server forms a micro-batch training set based on a sliding time window segmentation aggregation for parameter updating of an incremental learning engine, retains historical knowledge through knowledge distillation, and returns to the workstation for deployment after verification; the workstation monitors data distribution, triggers parameter reconstruction when concept drift is detected, and uses associated data as a seed data set, thereby realizing continuous intelligent self-adaptation of the cloud desktop workstation and improving real-time response capability and stability of the system to dynamic data distribution.

Description

Technical Field

[0001] This invention relates to the fields of data processing and cloud computing technology, and in particular to a method and system for intelligent linkage between cloud desktop workstations and big data servers. Background Technology

[0002] In cloud desktop workstation operational scenarios, big data servers typically handle the core tasks of model training and parameter updates. As the front-end node for data collection, the cloud desktop workstation continuously generates streaming data relevant to business operations. The current conventional approach is to upload the raw data collected by the workstations to a big data server, where the server performs offline or batch training to generate new model parameters before distributing them back to the workstations for deployment. This approach emphasizes centralized data processing and periodic model updates to maintain the system's adaptability to business changes.

[0003] Conventional incremental learning schemes often employ fixed-window data aggregation strategies, where the server fine-tunes the existing model or retrains it directly after receiving a batch of new data. To mitigate the knowledge forgetting problem during model parameter updates, some methods introduce regularization terms or replay mechanisms to maintain the model's response to older data distributions by retaining some historical samples or constraining the variation of important parameters. However, these approaches often focus on stability control within the parameter space, neglecting the direct impact of the distribution fluctuations of streaming data on model applicability. When data statistical features shift significantly, model performance deteriorates sharply, and existing anomaly detection mechanisms mostly rely on static thresholds or post-hoc backtracking, making it difficult to capture concept drift in real time.

[0004] Another common approach is to pre-set multiple candidate models on the server side or use an ensemble learning framework, automatically switching response strategies by evaluating the prediction errors of each model online. While this method enhances the system's robustness to data changes, the switching process typically requires significant computing resources and involves a high-latency model evaluation process, making it impossible to establish an efficient collaborative update channel between cloud desktop workstations and big data servers. Furthermore, data transmission between workstations and servers often uses a synchronous push model, making the data stream susceptible to network fluctuations during transmission. This can lead to outdated data sequence or uneven batches received by the server, thereby interfering with the convergence of incremental learning and the validity verification of parameter updates.

[0005] Existing technologies generally lack real-time monitoring methods for the local data distribution characteristics of workstations, and also fail to dynamically bind concept drift signals to the parameter reconstruction process. The deployment unit that sends the updated model back to the workstation often requires a long verification period, during which the data distribution has already undergone new changes, leaving the workstation in a passive adaptive state and making it difficult to achieve intelligent linkage and adaptive control between the cloud and the terminal. Summary of the Invention

[0006] This invention provides a method and system for intelligent linkage between cloud desktop workstations and big data servers, which can solve the problems in the prior art.

[0007] A first aspect of this invention provides a method for intelligent linkage between a cloud desktop workstation and a big data server, comprising:

[0008] The cloud desktop workstation continuously collects streaming data generated during business operations, performs time-series labeling and modality recognition on the streaming data, and generates data stream units with timestamps and modality tags.

[0009] The data stream unit is pushed to the big data server through an asynchronous transmission channel, and the big data server performs segmented aggregation of the data stream unit based on a sliding time window to form a micro-batch training set.

[0010] Based on the micro-batch training set, the parameters of the pre-deployed incremental learning engine are updated. During the parameter update process, a historical knowledge retention mechanism is introduced. By constructing a knowledge distillation constraint term, the incremental learning engine can maintain its responsiveness to historical data distribution while fusing new data features, thus obtaining the updated parameter configuration.

[0011] The updated parameter configuration is validated. When the validation indicators meet the preset stability conditions, the updated parameter configuration is encapsulated into a deployable unit and sent back to the cloud desktop workstation.

[0012] The deployable unit is deployed on the cloud desktop workstation, and the statistical feature distribution of the data stream unit is monitored in real time. When the data distribution is detected to deviate from the applicable scope of the incremental learning engine, a concept drift signal is generated to trigger the big data server to start the parameter reconstruction process. At the same time, the data stream unit associated with the concept drift signal is used as the seed dataset for parameter reconstruction.

[0013] The system continuously collects streaming data generated during business operations on cloud desktop workstations, performs time-series labeling and modality recognition on the streaming data, and generates data stream units with timestamps and modality tags, including:

[0014] By injecting hooks into the runtime interface of the business application through the streaming data capture component deployed on the cloud desktop workstation, the raw data stream output by the business application is intercepted in real time, and a timestamp containing the acquisition time and a globally incrementing sequence number is generated for each data segment in the raw data stream, forming a time-series labeled data segment sequence.

[0015] Modal features are extracted from the time-annotated data segment sequence. Modal recognition vectors are constructed based on the encoding format and semantic structure of the data segments. The modal recognition vectors are matched with a preset modal feature library for similarity. Corresponding modal labels are assigned to the successfully matched data segments to obtain time-series data segments with modal labels.

[0016] The time-series data segments with modality labels are sorted and aggregated based on the globally incrementing sequence number in the timestamp, and time-series data segments of the same modality category are grouped into the same data stream unit according to the modality label, while the timestamp is used as the time-series index of the data stream unit.

[0017] By injecting hooks into the runtime interface of business applications through a streaming data capture component deployed on a cloud desktop workstation, the raw data stream output by the business applications is intercepted in real time, and a timestamp containing the acquisition time and a globally incrementing sequence number is generated for each data segment in the raw data stream, including:

[0018] A dynamic hook injection engine is deployed in the system call layer of the cloud desktop workstation. The dynamic hook injection engine identifies the symbolic address of the data output interface by scanning the memory image of the business application, and inserts interception jump code before the instruction sequence corresponding to the symbolic address, so that the data output operation of the business application triggers the callback processing of the streaming data capture component, thereby realizing the real-time capture of the original data stream.

[0019] When the callback processing of the streaming data capture component is triggered, the data content to be output is read from the output buffer of the business application and the data content is divided into data fragments according to the preset fragmentation rules. At the same time, the nanosecond-level clock service of the cloud desktop workstation is called to obtain the acquisition time, and a global incrementing sequence number is extracted from the atomic counter maintained in shared memory. The atomic counter automatically increments after each extraction operation to ensure the uniqueness and monotonicity of the global incrementing sequence number. The acquisition time and the global incrementing sequence number are combined to construct a timestamp.

[0020] The data stream units are pushed to the big data server via an asynchronous transmission channel, and the big data server performs segmented aggregation of the data stream units based on a sliding time window to form a micro-batch training set, including:

[0021] An asynchronous transmission buffer pool is established on the cloud desktop workstation. Data stream units are partitioned and stored according to modal tags, and an independent transmission priority queue is maintained for each partition. Data stream units are extracted from each partition in sequence according to the weight allocation strategy of the transmission priority queue and assembled into transmission batches. The transmission batches are pushed to the data receiving end of the big data server through the asynchronous transmission channel.

[0022] After parsing the transmission batch to obtain data stream units at the data receiving end, the timestamp carried by the data stream unit is extracted to construct a time index table. The window boundary of the sliding time window is initialized based on the time index table. The window start boundary is determined according to the earliest timestamp recorded in the time index table, and the window end boundary is determined according to the preset window span. Data stream units whose timestamps fall between the window start boundary and the window end boundary are included in the candidate aggregation set of the current window.

[0023] The data stream units in the candidate aggregation set are grouped according to modality labels, and a time-series linked list of data stream units is constructed in each group according to the order of timestamps. The data distribution features of each modality category within the current sliding time window are extracted by traversing the time-series linked list. The data distribution features are used as aggregation weights to perform a weighted aggregation operation on the candidate aggregation set to generate a micro-batch training set.

[0024] Based on the micro-batch training set, the parameters of the pre-deployed incremental learning engine are updated. During the parameter update process, a historical knowledge retention mechanism is introduced. By constructing a knowledge distillation constraint term, the incremental learning engine maintains its responsiveness to historical data distributions while fusing new data features. The updated parameter configuration includes:

[0025] After receiving the micro-batch training set, the current parameter configuration is extracted from the parameter repository of the incremental learning engine and solidified as historical knowledge anchors. Based on the historical knowledge anchors, the knowledge distillation teacher mapping is instantiated, and inference operations are performed on the samples in the micro-batch training set to output historical knowledge response representations.

[0026] The micro-batch training set is input into the incremental learning engine to perform forward propagation to obtain the current learning response representation. The representation space distance between the current learning response representation and the historical knowledge response representation is calculated. A knowledge distillation constraint term is constructed based on the representation space distance. The degree of knowledge forgetting during the parameter update process is constrained by penalizing the representation drift amplitude. The knowledge distillation constraint term and the prediction error loss term calculated for the micro-batch training set are fused into a joint optimization objective.

[0027] The parameter configuration of the incremental learning engine is updated by gradient descent according to the joint optimization objective, and the updated parameter configuration is written into the parameter repository as the updated parameter configuration.

[0028] By penalizing the degree of knowledge forgetting during the update process of the drift amplitude constraint parameters, the knowledge distillation constraint term and the prediction error loss term calculated for the micro-batch training set are fused into a joint optimization objective, including:

[0029] The Euclidean distance between the current learning response representation and the historical knowledge response representation in the high-dimensional feature space is calculated as the representation drift amplitude. A nonlinear penalty mapping function based on the representation drift amplitude is constructed. By introducing convexity constraints, the penalty intensity increases at an accelerated rate with the increase of the representation drift amplitude. The nonlinear penalty mapping function is applied to the representation drift amplitude to obtain the knowledge forgetting penalty amount.

[0030] The knowledge forgetting penalty is multiplied by the historical knowledge importance coefficient to form a knowledge distillation constraint term. At the same time, the deviation between the predicted output and the true label of each sample in the micro-batch training set is aggregated and calculated. The prediction error loss term is obtained by applying a loss metric operator to the deviation.

[0031] The knowledge distillation constraint term and the prediction error loss term are dynamically weighted and combined. The fusion ratio of the two terms is adjusted in real time according to the relative relationship between the representation drift amplitude and the preset drift warning threshold. When the representation drift amplitude is close to the preset drift warning threshold, the fusion weight of the knowledge distillation constraint term is automatically increased to suppress the knowledge forgetting rate and generate a joint optimization objective.

[0032] When a deviation in data distribution from the applicable scope of the incremental learning engine is detected, a concept drift signal is generated to trigger the big data server to initiate a parameter reconstruction process. Simultaneously, the data stream unit associated with the concept drift signal is used as the seed dataset for parameter reconstruction, including:

[0033] A distribution consistency verification unit is deployed on the big data server to extract statistical moment features from the micro-batch training set to construct the current distribution profile. The current distribution profile is then compared with the baseline distribution profile recorded during the training phase of the incremental learning engine. When the topological distance measurement exceeds the preset applicable range boundary, it is determined that a data distribution deviation has occurred. A concept drift signal is generated based on the data distribution deviation.

[0034] The concept drift signal is parsed to extract the data stream unit set identifier. Based on the data stream unit set identifier, the corresponding data stream unit is retrieved from the persistent storage area of ​​the data receiving end. The data stream units are arranged in time sequence according to their timestamps within the sliding time window to form a time-series associated data stream unit sequence. The data stream unit sequence is used as the seed dataset for parameter reconstruction.

[0035] A second aspect of this invention provides an intelligent linkage system between a cloud desktop workstation and a big data server, comprising:

[0036] The streaming annotation unit is used to continuously collect streaming data generated during business operations on the cloud desktop workstation, and to perform time-series annotation and modality recognition on the streaming data to generate data stream units with timestamps and modality tags.

[0037] The segmented aggregation unit is used to push the data stream unit to the big data server through an asynchronous transmission channel, and the big data server performs segmented aggregation on the data stream unit based on a sliding time window to form a micro-batch training set.

[0038] The parameter update unit is used to update the parameters of the pre-deployed incremental learning engine based on the micro-batch training set. During the parameter update process, a historical knowledge retention mechanism is introduced. By constructing a knowledge distillation constraint term, the incremental learning engine can maintain its responsiveness to historical data distribution while fusing new data features, and obtain the updated parameter configuration.

[0039] The verification and encapsulation unit is used to verify the validity of the updated parameter configuration. When the verification indicators meet the preset stability conditions, the updated parameter configuration is encapsulated into a deployable unit and sent back to the cloud desktop workstation.

[0040] The drift detection unit is used to deploy the deployable unit on the cloud desktop workstation and monitor the statistical feature distribution of the data stream unit in real time. When the data distribution is detected to deviate from the applicable scope of the incremental learning engine, a concept drift signal is generated to trigger the big data server to start the parameter reconstruction process. At the same time, the data stream unit associated with the concept drift signal is used as the seed dataset for parameter reconstruction.

[0041] A third aspect of the present invention provides an electronic device, comprising:

[0042] processor;

[0043] Memory used to store processor-executable instructions;

[0044] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0045] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0046] Asynchronous data transmission from cloud desktop workstations to big data servers, combined with sliding time window segmented aggregation, significantly reduces network bandwidth consumption and transmission latency, enabling efficient real-time processing of streaming data. Dynamic generation of micro-batch training sets allows the incremental learning engine to continuously absorb new features from business operations, synchronously updating model parameters without service interruption or waiting for full data accumulation. The introduction of knowledge distillation constraints effectively suppresses catastrophic forgetting, maintaining a stable response to historical data distribution while fusing new data features, significantly improving the robustness of long-term model deployment.

[0047] The validity verification mechanism after parameter updates ensures that the model meets preset stability conditions after each update, avoiding the risk of unstable parameters introducing risks into the business system. After being packaged into deployable units, the models are uploaded back to cloud desktop workstations for direct application in local inference, significantly shortening the model iteration cycle and reducing continuous dependence on cloud computing resources. The ability to monitor the statistical feature distribution of data stream units in real time enables the system to proactively detect changes in the business environment. Once a data distribution deviation from the model's applicable range is detected, a concept drift signal is immediately generated to trigger the parameter reconstruction process.

[0048] The seed dataset associated with concept drift signals is used directly as the starting point for reconstruction, avoiding redundant data collection and accelerating the model retraining process. The entire integrated approach achieves closed-loop automation from data collection, model updates, validation feedback, to monitoring and reconstruction, significantly reducing the need for manual intervention and improving the maintainability of enterprise applications. The combination of incremental learning and asynchronous transmission ensures that the computing resources required for each round of system updates are allocated on demand, avoiding overload of big data servers, while the lightweight deployment of cloud desktop workstations guarantees fast front-end response. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating the intelligent linkage method between a cloud desktop workstation and a big data server according to an embodiment of the present invention.

[0050] Figure 2 This is a flowchart illustrating the parameter reconstruction process based on concept drift signals in an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0053] Figure 1 This is a flowchart illustrating the intelligent linkage method between a cloud desktop workstation and a big data server according to an embodiment of the present invention.

[0054] The intelligent linkage methods between cloud desktop workstations and big data servers include:

[0055] The cloud desktop workstation continuously collects streaming data generated during business operations, performs time-series labeling and modality recognition on the streaming data, and generates data stream units with timestamps and modality tags.

[0056] The data stream unit is pushed to the big data server through an asynchronous transmission channel, and the big data server performs segmented aggregation of the data stream unit based on a sliding time window to form a micro-batch training set.

[0057] Based on the micro-batch training set, the parameters of the pre-deployed incremental learning engine are updated. During the parameter update process, a historical knowledge retention mechanism is introduced. By constructing a knowledge distillation constraint term, the incremental learning engine can maintain its responsiveness to historical data distribution while fusing new data features, thus obtaining the updated parameter configuration.

[0058] The updated parameter configuration is validated. When the validation indicators meet the preset stability conditions, the updated parameter configuration is encapsulated into a deployable unit and sent back to the cloud desktop workstation.

[0059] The deployable unit is deployed on the cloud desktop workstation, and the statistical feature distribution of the data stream unit is monitored in real time. When the data distribution is detected to deviate from the applicable scope of the incremental learning engine, a concept drift signal is generated to trigger the big data server to start the parameter reconstruction process. At the same time, the data stream unit associated with the concept drift signal is used as the seed dataset for parameter reconstruction.

[0060] In one optional implementation, the cloud desktop workstation continuously collects streaming data generated during business operations, and performs time-series labeling and modality recognition on the streaming data to generate data stream units with timestamps and modality tags, including:

[0061] By injecting hooks into the runtime interface of the business application through the streaming data capture component deployed on the cloud desktop workstation, the raw data stream output by the business application is intercepted in real time, and a timestamp containing the acquisition time and a globally incrementing sequence number is generated for each data segment in the raw data stream, forming a time-series labeled data segment sequence.

[0062] Modal features are extracted from the time-annotated data segment sequence. Modal recognition vectors are constructed based on the encoding format and semantic structure of the data segments. The modal recognition vectors are matched with a preset modal feature library for similarity. Corresponding modal labels are assigned to the successfully matched data segments to obtain time-series data segments with modal labels.

[0063] The time-series data segments with modality labels are sorted and aggregated based on the globally incrementing sequence number in the timestamp, and time-series data segments of the same modality category are grouped into the same data stream unit according to the modality label, while the timestamp is used as the time-series index of the data stream unit.

[0064] In the cloud desktop workstation operating environment, streaming data acquisition is not accomplished through passive polling, but rather by actively intercepting the runtime interfaces of business applications using hook injection technology. Specifically, the streaming data capture component registers during the business application startup phase and embeds the capture logic into the business application's output interface call chain through dynamic link library injection or API hook mechanisms provided by the operating system. When the business application outputs data externally, the hook function is triggered before or after the original call execution, copying the raw data stream to an independent data buffer. This achieves real-time interception of the raw data stream generated during the business application's operation without affecting the normal execution flow of the business application itself.

[0065] After data interception, each data segment in the original data stream needs to be time-stamped. The timestamp consists of two key fields: the acquisition time and a globally incrementing sequence number. The acquisition time records the precise point in time when the data segment was captured, typically using a system clock with nanosecond or millisecond precision to ensure time-series differentiation between different data segments. The globally incrementing sequence number is generated by a monotonically increasing counter maintained by the capture component; the counter increments by one for each captured data segment, ensuring unique sorting by sequence number even when multiple data segments appear at the same acquisition time. These two fields together constitute the timestamp, forming the time-stamped sequence of data segments, providing a reliable order basis for subsequent sorting and aggregation operations.

[0066] When extracting modal features from time-annotated data segments, it is necessary to construct modal recognition vectors based on the structural information of the data segments themselves. The encoding format of a data segment refers to its binary or character encoding representation, such as JSON, binary protocol, or XML formats. These format features can be identified by parsing the header bytes or specific flags of the data segment. Semantic structure refers to the organization and type distribution of fields within the data segment, such as whether it contains time-related fields, numeric fields, text fields, and the nesting relationships between fields. By jointly encoding the encoding format features and semantic structure features, a modal recognition vector that can characterize the modal attributes of the data segment can be constructed.

[0067] After the modality recognition vector is constructed, it is matched with reference vectors stored in a pre-defined modality feature library for similarity. The modality feature library pre-stores standard feature vectors for various known modalities, with each record corresponding to a modality category and its typical feature representation. During the similarity matching process, the cosine similarity or Euclidean distance between the vector to be identified and each reference vector in the library is calculated. The modality category corresponding to the reference vector with the highest similarity exceeding a pre-defined matching threshold is taken as the matching result, and this modality label is assigned to the current data segment. If the similarity of all reference vectors is lower than the matching threshold, the data segment is marked as an unknown modality and enters the process of manual review or expansion of the modality feature library to ensure the continuous improvement of the modality feature library.

[0068] After modality recognition, each data segment carries two types of metadata: a timestamp and a modality label. Based on this, all data segments are sorted in ascending order according to the globally increasing sequence number in the timestamps, ensuring that the data segment sequence strictly follows the order of acquisition and eliminating out-of-order issues caused by concurrent capture or buffer scheduling. After sorting, the data segments are merged according to their modality labels: consecutive or adjacent data segments with the same modality label are merged into the same data stream unit, forming a set of data stream units based on modality category. During the merging process, the data stream unit inherits the timestamp range of its contained data segments; that is, the timestamp of the first data segment is used as the starting time-series index, and the timestamp of the last data segment is used as the ending time-series index, thus giving the data stream unit a clear time-series positioning capability.

[0069] In real-world business scenarios, multiple data fragments of different modalities may coexist within the same time window. For example, a business application might generate structured log data and unstructured text data simultaneously while processing user requests. In such cases, the merging strategy needs to aggregate different types of data fragments separately based on modality tags, forming multiple parallel data stream units, rather than forcibly merging all data fragments into a single data stream unit. This method of independent merging by modality category allows the subsequent big data server to employ appropriate analysis strategies for different modalities when processing data stream units, improving overall processing efficiency and analysis accuracy.

[0070] Timestamps, serving as temporal indexes for data stream units, play multiple roles throughout the entire linkage process. Firstly, the temporal index provides the big data server with precise temporal positioning for segmented aggregation based on sliding time windows, ensuring that the boundaries of the sliding window are aligned with the actual data generation time. Secondly, during the concept drift detection phase, the temporal index helps accurately locate the time interval where distribution shifts occur, enabling the concept drift signal to carry effective temporal context information, facilitating precise delineation of the seed dataset's time range in subsequent parameter reconstruction processes. The dual recording mechanism of globally incrementing sequence numbers and acquisition times also provides a basis for integrity verification and deduplication when data fragments are lost or duplicated during transmission, ensuring the integrity and consistency of data stream units.

[0071] The stability of hook injection directly affects the continuity of streaming data acquisition. To prevent capture interruptions due to abnormal exits of business applications or failed API calls, the capture component has a heartbeat detection mechanism that periodically checks the hook injection status. Once a hook failure is detected, a re-injection process is automatically triggered, and the data gaps during the interruption are recorded in the exception log. This allows for marking of the gap intervals during subsequent data stream unit processing, preventing bias in model training due to missing data. This fault-tolerant design ensures that the entire streaming data acquisition chain maintains high data integrity even when facing unstable business application operation.

[0072] In one optional implementation, a hook injection is performed on the runtime interface of the business application by a streaming data capture component deployed on a cloud desktop workstation to intercept the raw data stream output by the business application in real time, and a timestamp containing the acquisition time and a globally incrementing sequence number is generated for each data segment in the raw data stream, including:

[0073] A dynamic hook injection engine is deployed in the system call layer of the cloud desktop workstation. The dynamic hook injection engine identifies the symbolic address of the data output interface by scanning the memory image of the business application, and inserts interception jump code before the instruction sequence corresponding to the symbolic address, so that the data output operation of the business application triggers the callback processing of the streaming data capture component, thereby realizing the real-time capture of the original data stream.

[0074] When the callback processing of the streaming data capture component is triggered, the data content to be output is read from the output buffer of the business application and the data content is divided into data fragments according to the preset fragmentation rules. At the same time, the nanosecond-level clock service of the cloud desktop workstation is called to obtain the acquisition time, and a global incrementing sequence number is extracted from the atomic counter maintained in shared memory. The atomic counter automatically increments after each extraction operation to ensure the uniqueness and monotonicity of the global incrementing sequence number. The acquisition time and the global incrementing sequence number are combined to construct a timestamp.

[0075] When deploying a dynamic hook injection engine at the system call layer of a cloud desktop workstation, an image scan of the target business application's process memory space is first required. During the scan, the dynamic hook injection engine locates the symbol addresses of interface functions related to data output by parsing the symbol tables of the executable files and their dependent libraries loaded by the business application. These symbol addresses correspond to the starting positions of the data output instruction sequences actually called by the business application at runtime. After the scan is complete, the dynamic hook injection engine writes interception jump code at the very beginning of the instruction sequence corresponding to the target symbol address. This jump code is usually implemented in the form of an unconditional jump instruction, forcibly transferring the program execution flow from the original data output path to the callback processing function entry point pre-registered by the streaming data capture component. To ensure that the injection process does not affect the normal operation logic of the business application, the interception jump code must also restore the execution of the original instructions after completing the data capture callback, so that the data output operations of the business application remain logically intact.

[0076] When performing memory image scanning, the dynamic hook injection engine needs to handle the issue of variable symbol addresses caused by address space layout randomization. To address this, the engine re-executes symbol address resolution each time the application starts or a module is loaded, and caches the resolution results in a local mapping table for subsequent write operations of the interceptor code. Furthermore, when the application experiences dynamic library unloading or hot updates, the engine can detect the corresponding memory mapping changes, promptly remove invalid interceptor code, and re-inject it, thus ensuring the hook injection mechanism remains effective throughout the entire lifecycle of the application.

[0077] When the callback processing of the streaming data capture component is triggered, the data content to be output is read from the output buffer of the business application. The output buffer stores the raw byte sequence passed by the business application when calling the data output interface; its length and format vary depending on the type of business application. After reading, the data content is segmented according to preset segmentation rules to form several independent data fragments. Segmentation rules can be configured based on various strategies such as fixed byte length, specific delimiters, or semantic boundaries to adapt to the structural characteristics of data streams in different business scenarios. For example, for log data streams, newline characters can be used as segmentation boundaries; for binary protocol data streams, segmentation can be performed according to the protocol frame header identifier. After the segmentation operation is completed, each data fragment serves as the basic processing unit for subsequent timestamp construction and modality recognition.

[0078] While segmenting the data, the system invokes the nanosecond-level clock service of the cloud desktop workstation to obtain the current acquisition time. The nanosecond-level clock service obtains the time value by reading a high-precision hardware clock counter maintained by the operating system kernel. Compared to millisecond-level clocks, it can more precisely distinguish data segments generated consecutively within extremely short time intervals, thus providing a higher-precision time basis for subsequent time series analysis and event reconstruction. The acquired acquisition time is represented as an integer timestamp with nanosecond precision, denoted as […]. ,in The unit is nanosecond, representing the number of nanoseconds that have elapsed since the system reference time point.

[0079] Parallel to the acquisition of data at each acquisition moment is the extraction of a globally incrementing sequence number, provided by an atomic counter maintained in a shared memory region. The use of shared memory allows multiple streaming data capture component instances running on the cloud desktop workstation to access the same counter, ensuring the global uniqueness of sequence number allocation in cross-process and cross-thread scenarios. The reading and incrementing operations of the atomic counter are completed through atomic instructions provided by the processor, ensuring that there are no duplicate or out-of-order sequence number allocations in high-concurrency callback-triggered scenarios. After each extraction operation, the value of the atomic counter is automatically incremented by one, recording the currently extracted globally incrementing sequence number as 1. ,in It is a non-negative integer and strictly monotonically increasing throughout the entire acquisition lifecycle. Its monotonicity ensures that even if there are slight jitters or rollbacks in the system clock, the generation order of data segments can be accurately restored by the size relationship of the sequence numbers.

[0080] Collection time With globally incrementing sequence number The combined structure is used to construct a complete timestamp, and the specific structure of the timestamp is designed to... As the primary sorting key, As a composite identifier serving as a secondary sorting key, this design leverages the high-precision time information of nanosecond-level clocks while using the monotonicity of the sequence number to compensate for sorting ambiguities arising from insufficient clock precision to distinguish extremely high-frequency data segments. The constructed timestamp, along with the corresponding data segment content, is encapsulated into the basic structure of a data stream unit, awaiting subsequent modality recognition processing to attach modality tags, ultimately forming a data stream unit with complete metadata.

[0081] In practical deployments, the injection operation of the dynamic hook injection engine requires corresponding system privileges. It typically runs as a service process with elevated privileges, and its target process range is restricted through security policy configurations to prevent accidental injection into non-business processes that could cause system stability issues. The callback processing functions of the streaming data capture component should be designed to be as lightweight as possible, offloading time-consuming data processing logic to an asynchronous queue to reduce the latency impact on business application data output operations. The shared memory area where the atomic counter resides is allocated and initialized by a dedicated initialization process when the cloud desktop workstation starts, and cleaned up when the system shuts down, ensuring that the lifecycle of the counter's state is consistent with the cloud desktop workstation's runtime cycle. Through the coordinated operation of these mechanisms, real-time, complete, and ordered capture and timestamping of raw data streams can be achieved without intruding on the business application's source code, providing a high-quality data foundation for subsequent incremental learning and concept drift detection.

[0082] In one optional implementation, the data stream unit is pushed to a big data server via an asynchronous transmission channel, and the big data server performs segmented aggregation of the data stream unit based on a sliding time window to form a micro-batch training set, including:

[0083] An asynchronous transmission buffer pool is established on the cloud desktop workstation. Data stream units are partitioned and stored according to modal tags, and an independent transmission priority queue is maintained for each partition. Data stream units are extracted from each partition in sequence according to the weight allocation strategy of the transmission priority queue and assembled into transmission batches. The transmission batches are pushed to the data receiving end of the big data server through the asynchronous transmission channel.

[0084] After parsing the transmission batch to obtain data stream units at the data receiving end, the timestamp carried by the data stream unit is extracted to construct a time index table. The window boundary of the sliding time window is initialized based on the time index table. The window start boundary is determined according to the earliest timestamp recorded in the time index table, and the window end boundary is determined according to the preset window span. Data stream units whose timestamps fall between the window start boundary and the window end boundary are included in the candidate aggregation set of the current window.

[0085] The data stream units in the candidate aggregation set are grouped according to modality labels, and a time-series linked list of data stream units is constructed in each group according to the order of timestamps. The data distribution features of each modality category within the current sliding time window are extracted by traversing the time-series linked list. The data distribution features are used as aggregation weights to perform a weighted aggregation operation on the candidate aggregation set to generate a micro-batch training set.

[0086] On the cloud desktop workstation side, an asynchronous transmission buffer pool is established for the continuous generation of multimodal data stream units to achieve data temporary storage and scheduling management. Within the buffer pool, data stream units are divided into several independent partitions according to modal tags. Each partition corresponds to a modal category, such as text, image, or structured table. Each partition is logically isolated from others and maintains its own independent transmission priority queue, thereby avoiding interference between different modal data within the buffer. Each priority queue is configured with corresponding weight parameters, the weight value being comprehensively set based on the business importance, time sensitivity, and data volume proportion of the modal data. When the accumulated data to be transmitted in the buffer pool reaches a preset batch trigger threshold, data stream units are extracted from each partition sequentially according to the weight allocation strategy of each partition's priority queue, assembling them into a transmission batch. Partitions with higher weights can extract more data stream units in the same round of scheduling, thus ensuring the timeliness of high-priority modal data transmission.

[0087] After the batch of data is assembled, it is pushed to the data receiving end of the big data server via an asynchronous transmission channel. The asynchronous transmission channel employs a non-blocking message queue mechanism, allowing the cloud desktop workstation to continue collecting and assembling the next batch without waiting for confirmation from the server after completing the batch push. This decouples sending and receiving, reducing end-to-end transmission latency. Upon receiving the batch, the data receiving end parses and reconstructs it to obtain the data stream unit sequence for each modal partition. During parsing, the timestamp field of each data stream unit is extracted, and all timestamps are sorted in ascending order and written into a time index table. The time index table stores the mapping relationship between the identifier of the data stream unit and its corresponding timestamp in key-value pairs, facilitating subsequent rapid location and retrieval based on the time dimension.

[0088] After the time index table is constructed, the window boundaries of the sliding time window are initialized based on the table. Let the earliest timestamp recorded in the time index table be... The preset window span is Then the starting boundary of the current sliding time window is The termination boundary is Any timestamp that meets the requirements All data stream units are included in the candidate aggregation set of the current window, among which Indicates the first The timestamp of each data stream unit. After completing the aggregation operation of the current window, the sliding time window increments by a preset step size. Slide backward, and the new window's starting boundary is updated. The termination boundary is updated accordingly. This process is repeated until the time span of all data stream units in the current transmission batch is covered. Step size Typically smaller than the window span This ensures that there are overlapping intervals between adjacent windows, guaranteeing that data stream units near the time boundary can be fully captured and avoiding information breakage caused by window cutting.

[0089] After the candidate aggregation set is determined, the data stream units within it are grouped according to modality labels, with data stream units of the same modality label belonging to the same group. Within each group, a time-series linked list is constructed based on the timestamps of each data stream unit, from smallest to largest. The nodes in the linked list are connected in chronological order, and the time interval information between adjacent nodes is recorded. By traversing the time-series linked list of each modality group, the data distribution characteristics of that modality category within the current sliding time window are extracted. The extraction of data distribution characteristics covers multiple dimensions: statistically analyzing the proportion of data stream units of that modality within the current window, calculating the mean and variance of the feature vectors of each data stream unit, and analyzing the fluctuation range of the feature differences between adjacent nodes in the time-series linked list, in order to comprehensively characterize the distribution state of that modality within the current time window.

[0090] Using the data distribution characteristics of each modality category as aggregation weights, a weighted aggregation operation is performed on the candidate aggregation set. Let the... The aggregation weight of each modal group within the current window is: The weighting is determined by a combination of the proportion of each modality's data and its feature stability. A higher proportion and smaller feature variance result in a greater weight, reflecting the representativeness and reliability of the modality's data within the current time window. During weighted aggregation, the feature sequences of the time-series linked lists within each modality group are weighted according to their respective characteristics. Weighted fusion is performed to generate the aggregated feature vector corresponding to the current sliding time window. The aggregated feature vectors of all sliding time windows are arranged in window order to form a micro-batch training set. Each sample in the micro-batch training set corresponds to the aggregation result of one time window, preserving the evolutionary information of multimodal data in the temporal dimension, while eliminating the interference of imbalance in the amount of data of each modality on the training process through weighted aggregation.

[0091] After the micro-batch training set is generated, it is passed to the incremental learning engine for parameter updates. Because the sliding window mechanism ensures the temporal continuity of the training set samples, the incremental learning engine can perceive the dynamic changes in the data stream over time when processing the micro-batch training set, which helps improve the model's responsiveness to recent business patterns. The partition priority scheduling of the asynchronous transmission buffer pool and the segmented aggregation mechanism of the sliding window work together to ensure the stability and data integrity of the data flow from cloud desktop workstations to big data servers in high-concurrency scenarios.

[0092] In one optional implementation, the parameters of a pre-deployed incremental learning engine are updated based on the micro-batch training set. A historical knowledge retention mechanism is introduced during the parameter update process. By constructing a knowledge distillation constraint term, the incremental learning engine maintains its responsiveness to historical data distributions while fusing new data features. The updated parameter configuration includes:

[0093] After receiving the micro-batch training set, the current parameter configuration is extracted from the parameter repository of the incremental learning engine and solidified as historical knowledge anchors. Based on the historical knowledge anchors, the knowledge distillation teacher mapping is instantiated, and inference operations are performed on the samples in the micro-batch training set to output historical knowledge response representations.

[0094] The micro-batch training set is input into the incremental learning engine to perform forward propagation to obtain the current learning response representation. The representation space distance between the current learning response representation and the historical knowledge response representation is calculated. A knowledge distillation constraint term is constructed based on the representation space distance. The degree of knowledge forgetting during the parameter update process is constrained by penalizing the representation drift amplitude. The knowledge distillation constraint term and the prediction error loss term calculated for the micro-batch training set are fused into a joint optimization objective.

[0095] The parameter configuration of the incremental learning engine is updated by gradient descent according to the joint optimization objective, and the updated parameter configuration is written into the parameter repository as the updated parameter configuration.

[0096] Upon receiving the micro-batch training set, the system first reads the currently stored parameter configuration from the incremental learning engine's parameter repository, copies it completely, and freezes it as a static copy. This copy does not participate in any gradient calculations during subsequent updates; it is used solely as a historical knowledge anchor. The historical knowledge anchor serves to preserve all the knowledge states accumulated by the incremental learning engine before encountering the current micro-batch training set, ensuring that the subsequently constructed knowledge distillation constraints accurately reflect the response patterns of historical data distributions. Based on this historical knowledge anchor, a knowledge distillation teacher map is instantiated. This involves constructing a teacher network with the same structure as the incremental learning engine but without updated parameters, using the frozen parameters as weights. This teacher network performs inference operations on each sample in the micro-batch training set, outputting a historical knowledge response representation. The historical knowledge response representation can be a feature vector generated by the teacher network at an intermediate or output layer. The specific layer chosen depends on the incremental learning engine's network structure design; typically, the output of the penultimate hidden layer, which has strong semantic expressive power, is selected to ensure sufficient information density in the representation.

[0097] The micro-batch training set is simultaneously input into the trainable incremental learning engine, and forward propagation is performed to obtain the current learned response representation. The current learned response representation and the historical knowledge response representation are structurally consistent, both being feature vectors of the same dimension, thus allowing for distance measurement within the same representation space. When calculating the representation space distance between the two, the mean squared error is used to quantify the representation difference of each sample. Let the i-th sample in the micro-batch training set... The historical knowledge response of each sample obtained through the teacher network is characterized as follows: The current learning response representation obtained by the incremental learning engine is as follows: Then, for this sample, the representation space distance Defined as ,in Represents the Euclidean norm. For all values ​​in the micro-batch training set... The mean distance between the representation spaces of each sample is used to obtain the knowledge distillation constraint term. ,Right now The physical meaning of this constraint is: when the incremental learning engine updates its parameters on new data, if its feature extraction behavior deviates too much from the historical knowledge anchor, then... This will result in a significant penalty, thereby suppressing the magnitude of parameter updates and preventing catastrophic forgetting.

[0098] When calculating the prediction error loss term for the micro-batch training set, the appropriate loss function is selected based on the nature of the specific business task. For classification tasks, cross-entropy loss is used; for regression tasks, mean squared error loss is used. Let the i-th... The true label for each sample is The prediction output of the incremental learning engine is Then the prediction error loss term In the regression scenario, it is represented as The knowledge distillation constraint and the prediction error loss term are fused into a joint optimization objective through a weighted summation. ,Right now ,in This is the knowledge distillation tradeoff coefficient, used to control the balance between the strength of historical knowledge retention and the ability to learn from new data. The value needs to be adjusted according to the rate of change of data distribution in the actual business scenario: when the business data distribution is relatively stable, it can be appropriately increased. To enhance the preservation of historical knowledge; when the data distribution changes significantly, the percentage should be appropriately reduced. This gives the incremental learning engine a stronger ability to adapt to new distributions.

[0099] When performing gradient descent updates on the parameter configuration of the incremental learning engine based on the joint optimization objective, for Calculate gradients for all trainable parameters of the incremental learning engine, and apply them according to the preset learning rate. Update the parameters along the negative gradient direction. Let the incremental learning engine be at the... The parameter vector before the secondary parameter update is: The updated parameter vector satisfy In practical engineering implementations, gradient descent can be performed using mini-batch stochastic gradient descent or an adaptive learning rate optimizer to improve the stability and convergence speed of parameter updates. It is important to note that the teacher network parameters corresponding to the historical knowledge anchor points remain frozen throughout the gradient calculation process and do not change with backpropagation. This design ensures that the knowledge distillation constraint terms are always penalized based on the same historical knowledge benchmark throughout the entire update process, avoiding constraint invalidation due to teacher network parameter drift.

[0100] After completing the gradient descent update on the training set in a single micro-batch, the resulting parameter vector will be... The updated parameter configuration is written to the parameter repository. The parameter repository employs a versioning strategy during writing, retaining snapshots of the most recent versions of the parameter configuration to support parameter rollback operations in case of validation failures during subsequent validity verification. After writing is complete, the parameter update process on the current micro-batch training set ends, and the process repeats upon the arrival of the next micro-batch training set, thus enabling online iterative parameter updates for the incremental learning engine in continuous data stream scenarios. The entire parameter update process, through the introduction of knowledge distillation constraints, effectively maintains responsiveness to historical data distributions while incorporating new data features, fundamentally solving the core challenge of catastrophic forgetting in continuous learning scenarios.

[0101] In one optional implementation, by penalizing the degree of knowledge forgetting during the update process of the drift amplitude constraint parameters, the knowledge distillation constraint term and the prediction error loss term calculated for the micro-batch training set are fused into a joint optimization objective, including:

[0102] The Euclidean distance between the current learning response representation and the historical knowledge response representation in the high-dimensional feature space is calculated as the representation drift amplitude. A nonlinear penalty mapping function based on the representation drift amplitude is constructed. By introducing convexity constraints, the penalty intensity increases at an accelerated rate with the increase of the representation drift amplitude. The nonlinear penalty mapping function is applied to the representation drift amplitude to obtain the knowledge forgetting penalty amount.

[0103] The knowledge forgetting penalty is multiplied by the historical knowledge importance coefficient to form a knowledge distillation constraint term. At the same time, the deviation between the predicted output and the true label of each sample in the micro-batch training set is aggregated and calculated. The prediction error loss term is obtained by applying a loss metric operator to the deviation.

[0104] The knowledge distillation constraint term and the prediction error loss term are dynamically weighted and combined. The fusion ratio of the two terms is adjusted in real time according to the relative relationship between the representation drift amplitude and the preset drift warning threshold. When the representation drift amplitude is close to the preset drift warning threshold, the fusion weight of the knowledge distillation constraint term is automatically increased to suppress the knowledge forgetting rate and generate a joint optimization objective.

[0105] like Figure 2 As shown, the method includes:

[0106] In incremental learning, knowledge forgetting is one of the core problems leading to model performance degradation. When the incremental learning engine receives a new micro-batch training set and updates its parameters, if reasonable constraints are not applied to the parameter changes, the feature representation space within the network will shift significantly, resulting in a rapid loss of responsiveness to historical data distributions. To address this, a non-linear penalty mechanism based on the magnitude of representation drift is constructed to quantitatively constrain the degree of knowledge forgetting during parameter updates.

[0107] For the first in the micro-batch training set Each sample is used to extract high-dimensional feature representation vectors through historical knowledge anchors and the current incremental learning engine. and Both vectors reside in the same feature space. The squared Euclidean distance between the two vectors is calculated as the magnitude of the representational drift of the sample. ,Right now This distance reflects the incremental learning engine's understanding of the first parameter under the current parameter state. The degree of deviation between the feature representation of a sample and the historical knowledge anchor. When When the value is small, it indicates that the current engine can still maintain a representational capability close to historical knowledge anchors; when... As it continues to increase, it means that historical knowledge is being covered by the learning process of new data.

[0108] In obtaining the magnitude of the characterization drift Next, a non-linear penalty mapping function is constructed. Acting on This is used to generate the corresponding knowledge forgetting penalty. This nonlinear penalty mapping function must satisfy the convexity constraint, i.e. This causes the penalty intensity to increase at an accelerating rate with the increase in the magnitude of the representation drift. A typical implementation is to use an exponential mapping, defined as follows: ,in This is the penalty sensitivity coefficient, which controls the acceleration of the penalty intensity as the drift amplitude increases. When When approaching zero, When the value approaches zero, the punitive effect is negligible; when When significantly increased, The exponential growth rate imposes strong constraints on parameter updates, effectively suppressing knowledge forgetting. The introduction of convexity constraints ensures that the penalty mechanism has a stronger ability to suppress large representation drifts, while remaining lenient on small-scale normal feature adjustments, avoiding excessive restrictions on the incremental learning engine's ability to fuse new data features.

[0109] After applying the nonlinear penalty mapping function to the representation drift magnitude of all samples in the micro-batch training set, The average penalty values ​​of each sample are aggregated to obtain the batch-level average knowledge forgetting penalty. Subsequently, Importance coefficient of historical knowledge Perform scalar multiplication to obtain the knowledge distillation constraint term. Importance coefficient of historical knowledge Pre-set parameters can be configured based on the knowledge retention requirements of the business scenario, especially for scenarios with high requirements for the stability of historical data distribution. Take the larger value; for scenarios that allow for rapid adaptation to new data distributions, Take the smaller value.

[0110] Meanwhile, the predicted output for each sample in the micro-batch training set With real labeling Apply a loss metric operator to the deviation between the two values ​​to calculate the prediction error loss term. The choice of loss metric operator depends on the task type. Classification tasks typically use cross-entropy loss, while regression tasks use mean squared error loss. The prediction error loss term reflects the degree to which the incremental learning engine fits the new data under the current parameter configuration and is the core source of gradients driving parameter updates towards the new data distribution.

[0111] After obtaining the knowledge distillation constraint terms With prediction error loss term Next, the two need to be dynamically weighted and combined to generate a joint optimization objective. Static, fixed-weight combinations cannot adapt to changes in the risk of knowledge forgetting at different stages. Therefore, a method based on the magnitude of representational drift and a preset drift warning threshold is introduced. A dynamic weighting mechanism for the relative relationships between them. The batch average is defined to represent the drift amplitude. and compare it with the preset drift warning threshold. Comparison and dynamic calculation of the fusion weights of knowledge distillation constraints. .

[0112] Specifically, when far below At that time, the risk of knowledge forgetting is lower. Taking a smaller value allows the prediction error loss term to dominate the parameter update direction, ensuring that the incremental learning engine fully absorbs new data features; when Approaching or exceeding At that time, the risk of knowledge forgetting increases significantly. Automatically increasing the constraint term in the knowledge distillation process enhances its inhibitory effect on parameter updates. A smooth, dynamic adjustment method employs a Sigmoid-type mapping, defined as follows: ,in This represents the upper limit of the fusion weight. To adjust the sensitivity coefficient, For the standard Sigmoid function. This mapping is in... much smaller When the output is close to zero, Exceed Post-output fast approach This enables an adaptive response to the risk of knowledge forgetting.

[0113] The final form of the joint optimization objective is: In each parameter update iteration, with For the parameter vector of the incremental learning engine Calculate the gradient and apply it to the learning rate. Perform gradient descent update to obtain the first... Round parameter vector. Dynamic weights. The joint optimization objective is recalculated after each microbatch is processed to ensure that it always reflects the knowledge forgetting risk status of the current learning stage. When multiple consecutive microbatch processes are completed... All remained at The following indicates that the incremental learning process is within a safe range, and parameter updates are mainly based on fitting new data; once... An upward trend has emerged and is approaching The dynamic weighting mechanism will intervene in advance, imposing constraints on parameter updates before large-scale knowledge forgetting occurs, thereby achieving proactive protection of the ability to respond to historical data distribution.

[0114] Through the synergistic effect of the aforementioned nonlinear penalty mapping and dynamic weighted fusion mechanism, the incremental learning engine can effectively maintain the memory of historical knowledge distribution while receiving new data, achieving a dynamic balance between the ability to learn new data and the strength of historical knowledge retention, and providing continuous and stable model update support for the intelligent linkage between cloud desktop workstations and big data servers.

[0115] In one optional implementation, when a deviation from the applicable scope of the incremental learning engine is detected in the data distribution, a concept drift signal is generated to trigger the big data server to initiate a parameter reconstruction process. Simultaneously, the data stream unit associated with the concept drift signal is used as the seed dataset for parameter reconstruction, including:

[0116] A distribution consistency verification unit is deployed on the big data server to extract statistical moment features from the micro-batch training set to construct the current distribution profile. The current distribution profile is then compared with the baseline distribution profile recorded during the training phase of the incremental learning engine. When the topological distance measurement exceeds the preset applicable range boundary, it is determined that a data distribution deviation has occurred. A concept drift signal is generated based on the data distribution deviation.

[0117] The concept drift signal is parsed to extract the data stream unit set identifier. Based on the data stream unit set identifier, the corresponding data stream unit is retrieved from the persistent storage area of ​​the data receiving end. The data stream units are arranged in time sequence according to their timestamps within the sliding time window to form a time-series associated data stream unit sequence. The data stream unit sequence is used as the seed dataset for parameter reconstruction.

[0118] A distribution consistency verification unit is deployed on the big data server to continuously monitor whether the statistical characteristics of the data stream units are consistent with the applicable scope of the incremental learning engine. For each arriving micro-batch training set, its statistical moment features are extracted to construct the current distribution profile. The extraction of statistical moment features covers first-order moments, second-order moments, and higher-order moments. These statistics characterize the distribution pattern of the data from different dimensions. The first-order moment reflects the central tendency of the data, the second-order moment describes the dispersion of the data and the correlation between dimensions, and the higher-order moments capture the asymmetry and tail thickness of the data distribution. Together, they constitute a multi-dimensional profile of the current data distribution. After the incremental learning engine completes each round of training, the statistical moment features of the training data used at that time are recorded as a baseline distribution profile and stored in the baseline profile library of the big data server as a reference baseline for subsequent distribution consistency comparisons.

[0119] The difference between the current distribution profile and the baseline distribution profile is quantified using a topological distance metric. This metric employs the Wasserstein distance to compare the two distribution profiles. This distance effectively measures the optimal transmission cost between two probability distributions. Compared to information-theoretic metrics such as KL divergence, the Wasserstein distance exhibits good numerical stability even when the distribution support sets do not overlap, making it suitable for real-time online distribution deviation detection scenarios. Let the probability distribution corresponding to the current distribution profile be... The probability distribution corresponding to the baseline distribution profile is: The Wasserstein distance between the two is denoted as The preset applicable scope boundary threshold is denoted as .when When the current data distribution deviates from the applicable scope of the incremental learning engine, the concept drift judgment logic is triggered.

[0120] To reduce the interference of transient noise on the judgment results, a continuous window confirmation mechanism is used to smooth the topological distance metric results. Specifically, in the continuous... In each micro-batch, if each micro-batch has All exceeded the threshold This formally confirms the occurrence of a data distribution deviation and generates a concept drift signal. The preset continuous confirmation window length is configured based on the rate of change of business data and the system's tolerance for response latency, typically set to 3 to 5 micro-batches. This mechanism effectively avoids false alarms caused by brief data fluctuations while ensuring timely response to genuine concept drift events.

[0121] The concept drift signal consists of multiple fields, including the timestamp of the drift occurrence, the micro-batch number sequence that triggered the drift determination, the corresponding data stream unit set identifier, and the topological distance metric. The data stream unit set identifier is a unique set of identifiers, each corresponding one-to-one with a specific data stream unit. It is assigned by the timing annotation module when the data stream unit is generated and is persistently stored in the data receiver's storage area along with the data stream unit. By parsing the data stream unit set identifier carried in the concept drift signal, the original data stream unit that triggered this drift determination can be precisely located.

[0122] The process of retrieving the corresponding data stream unit from persistent storage employs an index-accelerated mechanism. The data receiving end maintains a hash index table with the data stream unit identifier as the key and the storage address as the value. During retrieval, the physical storage location of the target data stream unit is directly located through hash lookup, avoiding the latency caused by a full table scan. For scenarios with large amounts of data, a Bloom filter can be used to pre-filter retrieval requests, quickly eliminating non-existent identifiers and further improving retrieval efficiency. After the retrieval is complete, the original content of all data stream units associated with the concept drift signal is obtained.

[0123] The retrieved data stream units need to be sequentially arranged according to their timestamps within the sliding time window to reconstruct the natural evolution order of the data stream. The temporal arrangement employs a stable sorting algorithm to sort the timestamp fields of the data stream units in ascending order. For data stream units with the same timestamp, their globally increasing sequence number is used as a secondary sorting key to ensure the determinism and uniqueness of the arrangement result. After arrangement, a temporally correlated sequence of data stream units is formed. This sequence fully preserves the temporal evolution information before and after the data distribution drift, providing training material with temporal context for subsequent parameter reconstruction.

[0124] Before being submitted to the parameter reconstruction process, the time-series associated data stream unit sequence must undergo an integrity check. This check includes verifying whether the timestamps of each data stream unit in the sequence are strictly monotonically increasing, and whether there are any missing or corrupted data stream unit contents. If a time series gap is detected, the gap is marked. During processing, the parameter reconstruction process will interpolate or segment the marked gap to avoid negatively impacting the reconstruction results.

[0125] After time-series arrangement and integrity verification, the data stream unit sequence is submitted to the big data server as a seed dataset for parameter reconstruction. The seed dataset not only contains the feature content of the original data stream units, but also carries the timestamp, modality label, and sliding time window number of each data stream unit. This allows the parameter reconstruction process to fully utilize the temporal structure and modal diversity of the data, and to make targeted parameter adjustments for the concept drift that has occurred. This enables the incremental learning engine to readjust to the current data distribution and restore its ability to effectively model business data.

[0126] A second aspect of this invention provides an intelligent linkage system between a cloud desktop workstation and a big data server, comprising:

[0127] The streaming annotation unit is used to continuously collect streaming data generated during business operations on the cloud desktop workstation, and to perform time-series annotation and modality recognition on the streaming data to generate data stream units with timestamps and modality tags.

[0128] The segmented aggregation unit is used to push the data stream unit to the big data server through an asynchronous transmission channel, and the big data server performs segmented aggregation on the data stream unit based on a sliding time window to form a micro-batch training set.

[0129] The parameter update unit is used to update the parameters of the pre-deployed incremental learning engine based on the micro-batch training set. During the parameter update process, a historical knowledge retention mechanism is introduced. By constructing a knowledge distillation constraint term, the incremental learning engine can maintain its responsiveness to historical data distribution while fusing new data features, and obtain the updated parameter configuration.

[0130] The verification and encapsulation unit is used to verify the validity of the updated parameter configuration. When the verification indicators meet the preset stability conditions, the updated parameter configuration is encapsulated into a deployable unit and sent back to the cloud desktop workstation.

[0131] The drift detection unit is used to deploy the deployable unit on the cloud desktop workstation and monitor the statistical feature distribution of the data stream unit in real time. When the data distribution is detected to deviate from the applicable scope of the incremental learning engine, a concept drift signal is generated to trigger the big data server to start the parameter reconstruction process. At the same time, the data stream unit associated with the concept drift signal is used as the seed dataset for parameter reconstruction.

[0132] A third aspect of the present invention provides an electronic device, comprising:

[0133] processor;

[0134] Memory used to store processor-executable instructions;

[0135] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0136] A fourth aspect of the present invention provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0137] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent linkage between cloud desktop workstations and big data servers, characterized in that, include: The cloud desktop workstation continuously collects streaming data generated during business operations, performs time-series labeling and modality recognition on the streaming data, and generates data stream units with timestamps and modality tags. The data stream unit is pushed to the big data server through an asynchronous transmission channel, and the big data server performs segmented aggregation of the data stream unit based on a sliding time window to form a micro-batch training set. Based on the micro-batch training set, the parameters of the pre-deployed incremental learning engine are updated. During the parameter update process, a historical knowledge retention mechanism is introduced. By constructing a knowledge distillation constraint term, the incremental learning engine can maintain its responsiveness to historical data distribution while fusing new data features, thus obtaining the updated parameter configuration. The updated parameter configuration is validated. When the validation indicators meet the preset stability conditions, the updated parameter configuration is encapsulated into a deployable unit and sent back to the cloud desktop workstation. The deployable unit is deployed on the cloud desktop workstation, and the statistical feature distribution of the data stream unit is monitored in real time. When the data distribution is detected to deviate from the applicable scope of the incremental learning engine, a concept drift signal is generated to trigger the big data server to start the parameter reconstruction process. At the same time, the data stream unit associated with the concept drift signal is used as the seed dataset for parameter reconstruction.

2. The method according to claim 1, characterized in that, The system continuously collects streaming data generated during business operations on cloud desktop workstations, performs time-series labeling and modality recognition on the streaming data, and generates data stream units with timestamps and modality tags, including: By injecting hooks into the runtime interface of the business application through the streaming data capture component deployed on the cloud desktop workstation, the raw data stream output by the business application is intercepted in real time, and a timestamp containing the acquisition time and a globally incrementing sequence number is generated for each data segment in the raw data stream, forming a time-series labeled data segment sequence. Modal features are extracted from the time-annotated data segment sequence. Modal recognition vectors are constructed based on the encoding format and semantic structure of the data segments. The modal recognition vectors are matched with a preset modal feature library for similarity. Corresponding modal labels are assigned to the successfully matched data segments to obtain time-series data segments with modal labels. The time-series data segments with modality labels are sorted and aggregated based on the globally incrementing sequence number in the timestamp, and time-series data segments of the same modality category are grouped into the same data stream unit according to the modality label, while the timestamp is used as the time-series index of the data stream unit.

3. The method according to claim 2, characterized in that, By injecting hooks into the runtime interface of business applications through a streaming data capture component deployed on a cloud desktop workstation, the raw data stream output by the business applications is intercepted in real time, and a timestamp containing the acquisition time and a globally incrementing sequence number is generated for each data segment in the raw data stream, including: A dynamic hook injection engine is deployed in the system call layer of the cloud desktop workstation. The dynamic hook injection engine identifies the symbolic address of the data output interface by scanning the memory image of the business application, and inserts interception jump code before the instruction sequence corresponding to the symbolic address, so that the data output operation of the business application triggers the callback processing of the streaming data capture component, thereby realizing the real-time capture of the original data stream. When the callback processing of the streaming data capture component is triggered, the data content to be output is read from the output buffer of the business application and the data content is divided into data fragments according to the preset fragmentation rules. At the same time, the nanosecond-level clock service of the cloud desktop workstation is called to obtain the acquisition time, and a global incrementing sequence number is extracted from the atomic counter maintained in shared memory. The atomic counter automatically increments after each extraction operation to ensure the uniqueness and monotonicity of the global incrementing sequence number. The acquisition time and the global incrementing sequence number are combined to construct a timestamp.

4. The method according to claim 1, characterized in that, The data stream units are pushed to the big data server via an asynchronous transmission channel, and the big data server performs segmented aggregation of the data stream units based on a sliding time window to form a micro-batch training set, including: An asynchronous transmission buffer pool is established on the cloud desktop workstation. Data stream units are partitioned and stored according to modal tags, and an independent transmission priority queue is maintained for each partition. Data stream units are extracted from each partition in sequence according to the weight allocation strategy of the transmission priority queue and assembled into transmission batches. The transmission batches are pushed to the data receiving end of the big data server through the asynchronous transmission channel. After parsing the transmission batch to obtain data stream units at the data receiving end, the timestamp carried by the data stream unit is extracted to construct a time index table. The window boundary of the sliding time window is initialized based on the time index table. The window start boundary is determined according to the earliest timestamp recorded in the time index table, and the window end boundary is determined according to the preset window span. Data stream units whose timestamps fall between the window start boundary and the window end boundary are included in the candidate aggregation set of the current window. The data stream units in the candidate aggregation set are grouped according to modality labels, and a time-series linked list of data stream units is constructed in each group according to the order of timestamps. The data distribution features of each modality category within the current sliding time window are extracted by traversing the time-series linked list. The data distribution features are used as aggregation weights to perform a weighted aggregation operation on the candidate aggregation set to generate a micro-batch training set.

5. The method according to claim 1, characterized in that, Based on the micro-batch training set, the parameters of the pre-deployed incremental learning engine are updated. During the parameter update process, a historical knowledge retention mechanism is introduced. By constructing a knowledge distillation constraint term, the incremental learning engine maintains its responsiveness to historical data distributions while fusing new data features. The updated parameter configuration includes: After receiving the micro-batch training set, the current parameter configuration is extracted from the parameter repository of the incremental learning engine and solidified as historical knowledge anchors. Based on the historical knowledge anchors, the knowledge distillation teacher mapping is instantiated, and inference operations are performed on the samples in the micro-batch training set to output historical knowledge response representations. The micro-batch training set is input into the incremental learning engine to perform forward propagation to obtain the current learning response representation. The representation space distance between the current learning response representation and the historical knowledge response representation is calculated. A knowledge distillation constraint term is constructed based on the representation space distance. The degree of knowledge forgetting during the parameter update process is constrained by penalizing the representation drift amplitude. The knowledge distillation constraint term and the prediction error loss term calculated for the micro-batch training set are fused into a joint optimization objective. The parameter configuration of the incremental learning engine is updated by gradient descent according to the joint optimization objective, and the updated parameter configuration is written into the parameter repository as the updated parameter configuration.

6. The method according to claim 5, characterized in that, By penalizing the degree of knowledge forgetting during the update process of the drift amplitude constraint parameters, the knowledge distillation constraint term and the prediction error loss term calculated for the micro-batch training set are fused into a joint optimization objective, including: The Euclidean distance between the current learning response representation and the historical knowledge response representation in the high-dimensional feature space is calculated as the representation drift amplitude. A nonlinear penalty mapping function based on the representation drift amplitude is constructed. By introducing convexity constraints, the penalty intensity increases at an accelerated rate with the increase of the representation drift amplitude. The nonlinear penalty mapping function is applied to the representation drift amplitude to obtain the knowledge forgetting penalty amount. The knowledge forgetting penalty is multiplied by the historical knowledge importance coefficient to form a knowledge distillation constraint term. At the same time, the deviation between the predicted output and the true label of each sample in the micro-batch training set is aggregated and calculated. The prediction error loss term is obtained by applying a loss metric operator to the deviation. The knowledge distillation constraint term and the prediction error loss term are dynamically weighted and combined. The fusion ratio of the two terms is adjusted in real time according to the relative relationship between the representation drift amplitude and the preset drift warning threshold. When the representation drift amplitude is close to the preset drift warning threshold, the fusion weight of the knowledge distillation constraint term is automatically increased to suppress the knowledge forgetting rate and generate a joint optimization objective.

7. The method according to claim 1, characterized in that, When a deviation in data distribution from the applicable scope of the incremental learning engine is detected, a concept drift signal is generated to trigger the big data server to initiate a parameter reconstruction process. Simultaneously, the data stream unit associated with the concept drift signal is used as the seed dataset for parameter reconstruction, including: A distribution consistency verification unit is deployed on the big data server to extract statistical moment features from the micro-batch training set to construct the current distribution profile. The current distribution profile is then compared with the baseline distribution profile recorded during the training phase of the incremental learning engine. When the topological distance measurement exceeds the preset applicable range boundary, it is determined that a data distribution deviation has occurred. A concept drift signal is generated based on the data distribution deviation. The concept drift signal is parsed to extract the data stream unit set identifier. Based on the data stream unit set identifier, the corresponding data stream unit is retrieved from the persistent storage area of ​​the data receiving end. The data stream units are arranged in time sequence according to their timestamps within the sliding time window to form a time-series associated data stream unit sequence. The data stream unit sequence is used as the seed dataset for parameter reconstruction.

8. An intelligent linkage system between cloud desktop workstations and big data servers, used to implement the method as described in any one of claims 1-7, characterized in that, include: The streaming annotation unit is used to continuously collect streaming data generated during business operations on the cloud desktop workstation, and to perform time-series annotation and modality recognition on the streaming data to generate data stream units with timestamps and modality tags. The segmented aggregation unit is used to push the data stream unit to the big data server through an asynchronous transmission channel, and the big data server performs segmented aggregation on the data stream unit based on a sliding time window to form a micro-batch training set. The parameter update unit is used to update the parameters of the pre-deployed incremental learning engine based on the micro-batch training set. During the parameter update process, a historical knowledge retention mechanism is introduced. By constructing a knowledge distillation constraint term, the incremental learning engine can maintain its responsiveness to historical data distribution while fusing new data features, and obtain the updated parameter configuration. The verification and encapsulation unit is used to verify the validity of the updated parameter configuration. When the verification indicators meet the preset stability conditions, the updated parameter configuration is encapsulated into a deployable unit and sent back to the cloud desktop workstation. The drift detection unit is used to deploy the deployable unit on the cloud desktop workstation and monitor the statistical feature distribution of the data stream unit in real time. When the data distribution is detected to deviate from the applicable scope of the incremental learning engine, a concept drift signal is generated to trigger the big data server to start the parameter reconstruction process. At the same time, the data stream unit associated with the concept drift signal is used as the seed dataset for parameter reconstruction.

9. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 7.

Images