Distributed cooperative processing method and system based on event driving

Abstract

The invention provides a distributed cooperative processing method and system based on event driving, and relates to the technical field of data processing, and the method comprises the steps: analyzing event stream data, screening the event stream data into time sequence features and semantic features, and fusing codes to construct event multi-dimensional feature vectors; constructing a load situation prediction model on the basis, and generating a comprehensive load descriptor; grouping the processing nodes according to the descriptors and reconstructing a resource topology; mapping to an optimization space to generate a scaling decision scheme; and constructing a risk assessment function to realize co-evolution of the prediction model and resource configuration. The system processing efficiency can be effectively improved, the resource consumption is reduced, and the load fluctuation adaptability is enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to data processing technology, and particularly to an event-driven distributed collaborative processing method and system. BACKGROUND

[0002] With the rapid development of cloud computing, Internet of Things and distributed systems, event-driven architecture has become the mainstream mode for processing real-time data in large-scale distributed application scenarios. In an event-driven system, data streams from multiple event sources need to be processed in real time, and the system needs to dynamically adjust resource configuration according to the changes of event streams to ensure service quality. At present, distributed collaborative processing technology is widely used in financial transactions, smart cities, industrial automation and other fields. The event streams in these application scenarios have the characteristics of high concurrency, strong burstiness and complex semantics, which pose severe challenges to the real-time response capability and resource utilization efficiency of the system.

[0003] Traditional distributed event processing methods mainly rely on static resource configuration or simple threshold triggering mechanism for resource scheduling, which is difficult to adapt to complex and variable event stream characteristics.

[0004] The existing technology usually processes the timing characteristics and semantic characteristics separately, lacks effective feature fusion mechanism, and cannot accurately predict the load trend, especially when facing event streams with complex semantic association, the prediction accuracy decreases significantly.

[0005] Traditional resource scheduling methods often use fixed thresholds or simple feedback control strategies, and do not fully consider the uncertainty of prediction results. When the prediction model deviates, the system cannot adjust the resource configuration strategy in time, which easily leads to over-provisioning or insufficient resource supply, affecting the overall performance and cost-effectiveness of the system.

[0006] The existing technology lacks a collaborative optimization mechanism between the prediction model and the resource configuration. When the load pattern changes, the update of the prediction model and the adjustment of the resource configuration strategy are often separated, which cannot realize closed-loop feedback. The system is difficult to adapt to dynamic and variable event stream characteristics, especially in the load mutation scenario, which affects the stability and reliability of the system. SUMMARY

[0007] The embodiment of the present application provides an event-driven distributed collaborative processing method and system, which can solve the problems in the prior art.

[0008] In a first aspect of the embodiment of the present application, an event-driven distributed collaborative processing method is provided, comprising: analyzing event stream data generated by a distributed event source, filtering the event stream data into timing characteristics and semantic characteristics, using an autoencoder to fuse and encode the filtered characteristics, and constructing an event multi-dimensional feature vector; construct a load situation prediction model based on the event multi-dimensional feature vector, deduce an event arrival rate curve and a prediction confidence in a future time window by using the load situation prediction model, map and fuse the event arrival rate curve and semantic features in the event multi-dimensional feature vector to generate a comprehensive load descriptor; quantify a load peak period according to the comprehensive load descriptor, group processing nodes in a resource topology according to semantic processing requirements corresponding to the load peak period, and reconstruct node quantity and routing weight according to gradient distribution of the event arrival rate curve to generate a dynamic resource topology structure; map node distribution and routing weight of the dynamic resource topology structure to an optimization space, construct a constrained problem with resource utilization rate and processing delay as optimization objectives, evolve a resource redundancy threshold based on the prediction confidence, and generate a scaling decision scheme; construct a risk assessment function based on the resource redundancy threshold in the scaling decision scheme, quantify deviation risk of resource configuration and predicted load, trigger incremental optimization of the load situation prediction model when the deviation risk breaks through a tolerance boundary, and realize co-evolution of the prediction model and resource configuration.

[0009] analyze event stream data generated by a distributed event source, filter the event stream data into time sequence features and semantic features, fuse the filtered features by using a self-encoder, and construct an event multi-dimensional feature vector including: analyze original event stream generated by a distributed event source, extract timestamp information and event identification information of each event in the original event stream, construct time sequence features reflecting event arrival time distribution characteristics based on the timestamp information, and construct semantic features reflecting event types and processing requirements based on the event identification information; perform time axis segmentation processing on the time sequence features, divide the time sequence features into a plurality of time sequence feature subsegments according to a time window, calculate event arrival frequency and arrival interval variance in each time sequence feature subsegment, and form a time sequence feature statistic set; perform semantic space mapping on the semantic features, map different event types to vector representations in a unified semantic space, calculate distance metrics of each event type in the unified semantic space, identify event type clusters with similar processing requirements based on the distance metrics, and form a semantic feature clustering result; input the time sequence feature statistic set and the semantic feature clustering result into a feature fusion encoder, perform dimension reduction projection on the time sequence feature statistic set by using the feature fusion encoder, simultaneously perform embedding coding on the semantic feature clustering result, perform tensor splicing on the dimension reduction projected time sequence representation and the embedding coded semantic representation, and obtain an event multi-dimensional feature vector.

[0010] constructing a load situation prediction model based on the event multi-dimensional feature vector, deducing an event arrival rate curve and a prediction confidence in a future time window by using the load situation prediction model, and fusing the event arrival rate curve and semantic features in the event multi-dimensional feature vector to generate a comprehensive load descriptor, including: separating a time sequence feature component and a semantic feature component from the event multi-dimensional feature vector, performing multi-scale time window decomposition on the time sequence feature component, and extracting a macro time sequence mode reflecting long-term evolution and a micro time sequence mode reflecting short-term fluctuations; inputting the macro time sequence mode into a trend prediction unit to obtain a baseline trend curve by trend extrapolation, inputting the micro time sequence mode into a fluctuation prediction unit to obtain a fluctuation offset by fluctuation superposition, and superimposing the baseline trend curve and the fluctuation offset to generate an event arrival rate curve; calculating a prediction stability of the trend prediction unit based on the stationarity of the macro time sequence mode, calculating a prediction fluctuation degree of the fluctuation prediction unit based on the randomness of the micro time sequence mode, and fusing the prediction stability and the prediction fluctuation degree to obtain a prediction confidence corresponding to the event arrival rate curve; calculating a load change rate for the event arrival rate curve, identifying a load change period in which the load change rate exceeds a preset load threshold, extracting a semantic type distribution of historical events in the period from the semantic feature component, and selecting a dominant semantic type set based on resource consumption intensity; structuring and packaging the event arrival rate curve, the prediction confidence, a time identifier of the load change period, and the dominant semantic type set to generate a comprehensive load descriptor.

[0011] inputting the macro time sequence mode into a trend prediction unit to obtain a baseline trend curve by trend extrapolation, inputting the micro time sequence mode into a fluctuation prediction unit to obtain a fluctuation offset by fluctuation superposition, and superimposing the baseline trend curve and the fluctuation offset to generate an event arrival rate curve, including: establishing a fractal dimension mapping for the macro time sequence mode, quantifying complexity features of monotone increasing and decreasing trend segments in the macro time sequence mode, constructing a fractal predictor based on the complexity features, evolving and deducing the fractal predictor in a future time window to obtain a trend extension sequence corresponding to each trend segment, and generating a continuous baseline trend curve by optimizing the connection characteristics of trend segment boundaries through the fractal predictor; dual feature decoupling is performed on the micro time sequence pattern, the micro time sequence pattern is projected to a periodicity and randomness feature space, a fluctuation period and a fluctuation amplitude are quantified in the periodicity feature space, a periodic fluctuation feature is reconstructed by a fractal predictor, a periodic fluctuation sequence is obtained by iterative evolution of the reconstructed periodic fluctuation feature in a future time window, a fluctuation intensity description is established for the randomness feature space, a random fluctuation sequence is generated, and the periodic fluctuation sequence and the random fluctuation sequence are adaptively combined to obtain a fluctuation offset. The baseline trend curve and the fluctuation offset are fused in the time dimension to generate an event arrival rate curve containing trend evolution information and fluctuation feature information.

[0012] According to the comprehensive load descriptor, a load peak period is quantified, processing nodes in a resource topology are grouped in combination with semantic processing requirements corresponding to the load peak period, and node quantity and routing weight are reconstructed according to the gradient distribution of the event arrival rate curve to generate a dynamic resource topology structure including: An entropy value mapping is established for the event arrival rate curve in the comprehensive load descriptor, a time interval feature in which the event arrival rate exceeds a historical average value is quantified, an integral load quantity in each time interval is calculated, and a load peak period is determined based on the integral load quantity; A dominant semantic type set associated with the load peak period is mapped to an entropy value space, a matching entropy metric is constructed in combination with semantic ability labels of processing nodes in a resource topology, an organization structure of the processing nodes is reconstructed according to the matching entropy metric, and a matching node group is generated; An entropy value mapping is established for the event arrival rate curve to obtain a gradient distribution curve, a feature of a load sudden change moment in the gradient distribution curve is quantified, a resource demand increment is deduced based on an event arrival rate growth trend before the load sudden change moment, and the resource demand increment is converted into a node expansion quantity of each matching node group; Resource distribution of each matching node group is reconstructed according to the node expansion quantity, load saturation degrees of processing nodes in the matching node group are quantified, and routing weight is constructed based on reciprocals of the load saturation degrees; organization structure, node information, quantity distribution and routing weight of the matching node group are topologically encoded to generate a dynamic resource topology structure.

[0013] Node distribution and routing weight of the dynamic resource topology structure and the prediction confidence are mapped to an optimization space, a constraint problem with resource utilization rate and processing delay as optimization objectives is constructed, a resource redundancy threshold value is evolved based on the prediction confidence, and a scaling decision scheme is generated including: Identify the node position and connection relationship in the dynamic resource topology as a spectral clustering space, map the node distribution as a distribution feature vector in the spectral clustering space, construct a correlation matrix based on the routing weight, and embed the prediction confidence into the correlation matrix to obtain a space modulation parameter; Construct a double-objective optimization architecture in the spectral clustering space, represent the resource utilization rate by the distribution uniformity of the distribution feature vector, represent the processing delay by the distribution characteristics of the correlation matrix, perform spectral clustering combination on the double objective under the space modulation parameter to obtain a comprehensive optimization target, and set a constraint condition for the distribution feature vector and the correlation matrix in the double-objective optimization architecture; Construct a spectral clustering compensation term based on the prediction confidence, apply the spectral clustering compensation term to the comprehensive optimization target to generate a resource redundancy threshold, and adjust the constraint boundary of the double-objective optimization architecture according to the resource redundancy threshold; Solve the optimal resource configuration scheme in the spectral clustering space, compare the optimal resource configuration scheme with the current node distribution, determine the number of node expansion and contraction based on the comprehensive optimization target, convert the optimal resource configuration scheme into a routing weight distribution ratio according to the space modulation parameter, and generate a scaling decision scheme.

[0014] Construct a risk assessment function based on the resource redundancy threshold in the scaling decision scheme, quantify the deviation risk of resource configuration and predicted load, and trigger the incremental optimization of the load situation prediction model when the deviation risk breaks through the tolerance boundary, including: Construct a robustness space based on the scaling decision scheme, quantify the coupling characteristic of the resource redundancy threshold and the actual resource configuration amount as a resource redundancy coefficient in the robustness space, and construct an initial robustness evaluation benchmark based on the resource redundancy coefficient; Construct a risk assessment structure in the robustness space, map the difference between the resource redundancy coefficient and the safety redundancy benchmark as the configuration robustness, map the deviation between the actual event arrival rate and the predicted event arrival rate as the load robustness, construct a risk assessment function through the configuration robustness and the load robustness, and generate a deviation risk value in combination with the robustness evaluation benchmark; Set a robustness tolerance boundary based on the deviation risk value, dynamically adjust the robustness tolerance boundary using the characteristics of the robustness space, and reconstruct the error characteristics of the historical load sequence in the robustness space when the deviation risk value breaks through the robustness tolerance boundary; Generate a robustness optimization sample based on the reconstructed error characteristics, combine the evolution characteristics of the configuration robustness and the load robustness, and perform incremental optimization on the parameters of the load situation prediction model.

[0015] In a second aspect of the embodiment of the present application, a distributed cooperative processing system based on event driving is provided, comprising: A first unit is configured to parse event stream data generated by a distributed event source, filter the event stream data into timing features and semantic features, fuse and encode the filtered features by using a self-encoder, and construct an event multi-dimensional feature vector; A second unit is configured to construct a load situation prediction model based on the event multi-dimensional feature vector, deduce an event arrival rate curve and a prediction confidence in a future time window by using the load situation prediction model, map and fuse the event arrival rate curve and the semantic features in the event multi-dimensional feature vector to generate a comprehensive load descriptor; A third unit is configured to quantize a load peak period according to the comprehensive load descriptor, group processing nodes in a resource topology according to semantic processing requirements corresponding to the load peak period, and reconstruct the number of nodes and routing weights according to the gradient distribution of the event arrival rate curve to generate a dynamic resource topology structure; A fourth unit is configured to map the node distribution, routing weights and prediction confidence of the dynamic resource topology structure to an optimization space, construct a constraint problem with resource utilization rate and processing delay as optimization objectives, evolve a resource redundancy threshold based on the prediction confidence, and generate a scaling decision scheme; A fifth unit is configured to construct a risk assessment function based on the resource redundancy threshold in the scaling decision scheme, quantify the deviation risk of resource configuration and predicted load, trigger incremental optimization of the load situation prediction model when the deviation risk breaks through a tolerance boundary, and realize the co-evolution of the prediction model and resource configuration.

[0016] In a third aspect of the embodiment of the present application, an electronic device is provided, comprising: A processor; A memory for storing processor-executable instructions; The processor is configured to invoke the instructions stored in the memory to execute the method described above.

[0017] In a fourth aspect of the embodiment of the present application, a computer readable storage medium is provided, which stores computer program instructions, and the computer program instructions are executed by a processor to implement the method described above.

[0018] The present application has the following beneficial effects: The present application decomposes event stream data into timing features and semantic features, fuses and encodes the features by using a self-encoder to construct a multi-dimensional feature vector, and realizes accurate modeling of a complex event environment. The load situation prediction model constructed based on this can accurately deduce future event arrival rates and quantify prediction confidence, so that the system can perceive load changes in advance and make corresponding adjustments.

[0019] The application quantifies the load peak period by comprehensively describing the load descriptor, intelligently groups the processing nodes in combination with the semantic processing demand, and dynamically reconstructs the node number and routing weight according to the gradient distribution of the event arrival rate curve, thereby constructing a dynamic resource topology structure capable of adaptive adjustment, and significantly improving the system processing capacity and resource utilization efficiency.

[0020] The application introduces a prediction confidence driven resource redundancy threshold mechanism, constructs an optimization constraint model of resource utilization rate and processing delay, and realizes accurate control of the scaling decision. At the same time, through a risk assessment function, the deviation between resource allocation and predicted load is dynamically monitored, and when the risk breaks through the tolerance boundary, the incremental optimization of the load prediction model is triggered, realizing the co-evolution of the prediction model and resource allocation, and guaranteeing the long-term stability and efficiency of the system. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The figure is a flowchart of the event-driven distributed collaborative processing method of the embodiment of the application. Figure 2 The figure is a flowchart of the event-driven distributed collaborative processing method of the embodiment of the application. DETAILED DESCRIPTION

[0022] To make the purpose, technical scheme and advantages of the embodiments of the application clearer, the technical scheme in the embodiments of the application will be described clearly and completely below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the application.

[0023] The technical scheme of the application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes can not be described in some embodiments.

[0024] Figure 1 The figure is a flowchart of the event-driven distributed collaborative processing method of the embodiment of the application. Figure 1 As shown in the figure, the method comprises: Analyzing the event stream data generated by the distributed event source, filtering the event stream data into time sequence features and semantic features, using an autoencoder to fuse and encode the filtered features, and constructing an event multi-dimensional feature vector. construct a load situation prediction model based on the event multi-dimensional feature vector, deduce an event arrival rate curve and a prediction confidence in a future time window by using the load situation prediction model, fuse the event arrival rate curve and semantic features in the event multi-dimensional feature vector, and generate a comprehensive load descriptor; quantify a load peak period according to the comprehensive load descriptor, group processing nodes in a resource topology according to semantic processing requirements corresponding to the load peak period, and reconstruct the number of nodes and routing weights according to the gradient distribution of the event arrival rate curve, to generate a dynamic resource topology structure; map the node distribution, routing weights, and prediction confidence of the dynamic resource topology structure to an optimization space, construct a constraint problem with resource utilization rate and processing delay as optimization objectives, evolve a resource redundancy threshold based on the prediction confidence, and generate a scaling decision scheme; construct a risk assessment function based on the resource redundancy threshold in the scaling decision scheme, quantify the deviation risk of resource allocation and predicted load, trigger incremental optimization of the load situation prediction model when the deviation risk exceeds a tolerance boundary, and realize co-evolution of the prediction model and resource allocation.

[0025] In an optional implementation, event stream data generated by a distributed event source is parsed, and the event stream data is filtered into time sequence features and semantic features. The filtered features are fused and encoded by using a self-encoder to construct an event multi-dimensional feature vector, including: The original event stream generated by the distributed event source is parsed, timestamp information and event identification information of each event in the original event stream are extracted, time sequence features reflecting the distribution characteristics of event arrival time are constructed based on the timestamp information, and semantic features reflecting event types and processing requirements are constructed based on the event identification information; The time sequence features are subjected to time axis segmentation processing, the time sequence features are divided into a plurality of time sequence feature subsegments according to a time window, the event arrival frequency and arrival interval variance in each time sequence feature subsegment are calculated, and a time sequence feature statistic set is formed; The semantic features are subjected to semantic space mapping, different event types are mapped to vector representations in a unified semantic space, distance metrics of each event type in the unified semantic space are calculated, event type clusters with similar processing requirements are identified based on the distance metrics, and a semantic feature clustering result is formed; The time sequence feature statistic set and the semantic feature clustering result are input into a feature fusion encoder, the time sequence feature statistic set is subjected to dimension reduction projection by the feature fusion encoder, the semantic feature clustering result is subjected to embedding encoding at the same time, the time sequence representation after dimension reduction projection and the semantic representation after embedding encoding are spliced into a tensor, and an event multi-dimensional feature vector is obtained.

[0026] The raw event stream generated by the distributed event source is parsed by the event collection agent, which is deployed on each distributed node and listens to the data output of the local event source in real time. Each event collection agent maintains an event buffer queue, with a capacity of 10,000 event entries. When the queue usage rate exceeds 80%, the batch processing mechanism is triggered. Each event in the raw event stream contains three parts: event payload, metadata header, and control information. The event payload carries business data, the metadata header contains routing and processing instructions, and the control information records the life cycle status of the event.

[0027] The timestamp information extraction module reads the timestamp field from the metadata header of each event. The timestamp field is represented by a 64-bit integer, with a precision of microseconds and a time reference of Coordinated Universal Time. During the timestamp extraction process, clock skew and network delay issues need to be addressed. By deploying a network time protocol server, the clocks of each node are synchronized, with a synchronization accuracy of within 100 microseconds. For events with missing or abnormal timestamp formats, the time of arrival of the event is used as a substitute timestamp, and the source type of the timestamp is recorded in the event marker field.

[0028] The event identification information extraction module parses the event type field and processing priority field in the event payload. The event type field uses a hierarchical namespace structure, with a format of domain name plus type identifier combination. The processing priority field uses a 3-bit integer encoding, with a value range from 1 to 8, where 1 represents the highest priority and 8 represents the lowest priority. The event identification information also includes resource requirement tags, which use a key-value pair format to record the types and quantity estimates of computing resources required for event processing.

[0029] The time sequence feature construction process is based on the extracted timestamp information, which calculates the distribution characteristics of the event arrival time on the time axis. The time sequence features include arrival interval sequence, arrival frequency curve, and periodic pattern. The arrival interval sequence records the time difference between adjacent events, stored as a double-precision floating-point number with a unit of seconds. The arrival frequency curve calculates the number of events arriving within a unit of time through a sliding time window, with a window size of 60 seconds and a sliding step of 10 seconds. The periodic pattern identifies the periodic components in the arrival interval sequence through a fast Fourier transform, retaining the top 5 frequency components in the frequency domain with the largest amplitude as the periodic features.

[0030] The semantic feature construction process maps the event type and processing requirement to a structured semantic representation based on the extracted event identification information. The semantic feature includes three components: event type vector, processing requirement vector, and priority encoding. The event type vector uses one-hot encoding, and the vector dimension is determined according to the total number of event types in the system, usually set to 512 dimensions. The processing requirement vector is generated according to the resource requirement label, including CPU demand intensity, memory demand intensity, network bandwidth demand intensity, and storage demand intensity. Each component has a value range of 0 to 1 real number. The priority encoding converts the processing priority field into an 8-dimensional one-hot vector representation.

[0031] The timeline segmentation processing of the timing feature adopts a fixed time window division strategy, and the time window length is set to 300 seconds without overlapping between adjacent windows. The timing feature in each time window forms a timing feature subsegment, and the subsegment data structure includes four fields: window start time, window end time, event count, and statistical quantity array. Event count records the total number of events received in the time window, and the statistical quantity array stores the calculated statistical indicators in the window.

[0032] The event arrival frequency in each timing feature subsegment is calculated by dividing the event count by the time window length, with the unit of events per second. The arrival interval variance is calculated by collecting the time intervals of all adjacent events in the time window, and the variance value of the interval sequence. When the number of events in the time window is less than 3, the arrival interval variance is set to a predefined default value of 0.1. The timing feature statistical quantity set organizes the arrival frequency and arrival interval variance of each time window into a two-dimensional array structure, with the row dimension corresponding to the time window number and the column dimension corresponding to the statistical quantity type.

[0033] The semantic space mapping of the semantic feature uses a pre-trained word embedding model to convert the event type identifier into a dense vector representation in a unified semantic space. The word embedding model uses a 256-dimensional vector space and is trained on a large-scale event log corpus using an unsupervised learning method. The event type identifier is first preprocessed, including lowercase conversion, special character removal, and word segmentation, etc., and then the word embedding model is queried to obtain the corresponding vector representation. For event types that do not exist in the word embedding model, a randomly initialized 256-dimensional vector is used as a substitute representation, and the vector elements follow a normal distribution with a mean of 0 and a standard deviation of 0.1.

[0034] The distance metric between each event type in the unified semantic space adopts the cosine similarity calculation method, and the cosine similarity value ranges from -1 to 1. The closer the similarity is to 1, the more similar the two event types are. The distance metric matrix adopts an upper triangular matrix storage format to reduce memory usage and computational complexity. Based on the distance metric matrix, event type clusters with similar processing requirements are identified, and hierarchical clustering algorithm is used for clustering analysis. The clustering threshold is set to 0.7, that is, event types with a cosine similarity greater than 0.7 are grouped into the same cluster.

[0035] The semantic feature clustering result is represented by a data structure of cluster identifier and event type list in the cluster. The cluster identifier is an integer type, starting from 0 and increasing by 1. The representative semantic vector of each cluster is obtained by calculating the mean of all event type vectors in the cluster. The representative vector is used in the subsequent feature fusion encoding process. The clustering result also includes the tightness index of the cluster, which is obtained by calculating the average distance between the event type vector and the representative vector in the cluster. The tightness index is used to evaluate the clustering quality.

[0036] The feature fusion encoder adopts a multi-layer perceptron architecture, including an input layer, a hidden layer, and an output layer. The input layer receives two types of input data: the time series feature statistic set and the semantic feature clustering result. The number of input layer neurons is determined by the total dimension of the input features. The hidden layer adopts a two-layer fully connected network structure, with 512 neurons in the first hidden layer and 256 neurons in the second hidden layer. The activation function uses the rectified linear unit function. The number of output layer neurons is set to 128, corresponding to the dimension of the final event multi-dimensional feature vector.

[0037] The dimensionality reduction projection of the time series feature statistic set is processed by the first half of the input layer neurons. The dimensionality reduction projection is realized by the linear transformation of the principal component analysis method. The dimensionality reduction projection process maps the original time series feature statistics from high-dimensional space to 64-dimensional low-dimensional space, retaining more than 95% of the variance information in the original data. The transformation matrix of the dimensionality reduction projection is obtained by principal component analysis training on historical time series feature data. The transformation matrix is updated every 24 hours to adapt to changes in data distribution.

[0038] The embedding encoding of the semantic feature clustering result is processed by the second half of the input layer neurons. The embedding encoding converts discrete cluster identifiers into continuous vector representations. The embedding encoding is realized by a lookup table, which maintains a mapping table from cluster identifiers to vectors. The vector dimension is set to 64. The embedding vector is generated by random initialization and optimized and updated by the backpropagation algorithm during the training of the feature fusion encoder.

[0039] The time series representation after dimensionality reduction projection and the semantic representation after embedding encoding are subjected to a tensor splicing operation, and the splicing is along the feature dimension. The dimension of the spliced feature vector is 128. The tensor splicing operation is implemented as a connection operation in the calculation graph, and does not involve additional parameter learning. The spliced feature vector is subjected to linear transformation and normalization processing of the output layer to obtain the final event multi-dimensional feature vector. The normalization processing adopts an L2 norm normalization method to ensure that the length of the feature vector is 1.

[0040] A specific data case is used to illustrate the implementation process. It is assumed that a distributed event source generates 150 events in a 5-minute time window, and the event types include user login, data query, and file upload. The timestamp extraction obtains 150 timestamp values with microsecond precision, and the minimum interval of the timestamp sequence is 0.5 seconds and the maximum interval is 8.2 seconds. The event identification information extraction obtains 60 user login events, 70 data query events, and 20 file upload events, and the processing priority distribution is 45 high-priority events, 80 medium-priority events, and 25 low-priority events. The time series feature construction generates an arrival frequency of 0.5 events per second and an arrival interval variance of 2.3 seconds squared. The semantic feature construction maps the three event types into 256-dimensional vectors, and the cluster analysis classifies user login and data query into an interactive cluster and file upload into a separate cluster. The feature fusion encoder outputs a 128-dimensional event multi-dimensional feature vector, and the vector element value range is between -1 and 1, and the L2 norm is 1.

[0041] In an optional implementation, a load situation prediction model is constructed based on the event multi-dimensional feature vector, and an event arrival rate curve and a prediction confidence in a future time window are deduced using the load situation prediction model. The event arrival rate curve and the semantic feature in the event multi-dimensional feature vector are mapped and fused to generate a comprehensive load descriptor, which includes: The time series feature component and the semantic feature component are separated from the event multi-dimensional feature vector, the time series feature component is subjected to multi-scale time window decomposition, and macroscopic time series patterns reflecting long-term evolution and microscopic time series patterns reflecting short-term fluctuations are extracted. The macroscopic time series pattern is input into a trend prediction unit to obtain a baseline trend curve, and the microscopic time series pattern is input into a fluctuation prediction unit to obtain a fluctuation offset. The baseline trend curve and the fluctuation offset are superimposed to generate an event arrival rate curve. The prediction stability of the trend prediction unit is calculated based on the stationarity of the macroscopic time series pattern, and the prediction fluctuation degree of the fluctuation prediction unit is calculated based on the randomness of the microscopic time series pattern. The prediction stability and the prediction fluctuation degree are fused to obtain a prediction confidence corresponding to the event arrival rate curve. A load change rate is calculated for the event arrival rate curve, a load change period in which the load change rate exceeds a preset load threshold is identified, a semantic type distribution of historical events in the load change period is extracted from the semantic feature components, and a dominant semantic type set is selected based on resource consumption intensity; The event arrival rate curve, the prediction confidence, the time identifier of the load change period, and the dominant semantic type set are structured and encapsulated to generate a comprehensive load descriptor.

[0042] The separation processing of the event multi-dimensional feature vector is implemented by a feature decomposition module, which maintains a dimension mapping table of the feature vector. The mapping table records the position index of the time series feature component and the semantic feature component in the 128-dimensional feature vector. The time series feature component occupies the first 64 dimensions, corresponding to the time series representation after dimension reduction projection, and the semantic feature component occupies the last 64 dimensions, corresponding to the semantic representation after embedding coding. The feature decomposition module uses array slicing operation to extract the corresponding feature component, and the time complexity of the slicing operation is constant level, without data copying, only returning the view reference of the original array.

[0043] The multi-scale time window decomposition of the time series feature component uses wavelet transform technology to map the 64-dimensional time series feature component to the frequency domain representation of multiple time scales. The wavelet transform uses Daubechies wavelet basis function, and the decomposition layer is set to 4 layers to generate 5 frequency domain coefficient sets of different scales. The 0th layer corresponds to the highest frequency component, reflecting the short-term fluctuation characteristics of seconds, and the 4th layer corresponds to the lowest frequency component, reflecting the long-term evolution characteristics of hours. The boundary processing in the wavelet decomposition process uses the symmetric extension method to avoid the influence of boundary effects on the decomposition results.

[0044] The macroscopic time series pattern extraction process selects the 3rd and 4th layer wavelet coefficients as long-term evolution features. These two layers of coefficients reflect the time scale changes from minutes to hours. The macroscopic time series pattern data structure contains three fields: coefficient array, scale identifier, and time span. The coefficient array stores the numerical sequence obtained by wavelet transform, the scale identifier records the corresponding time scale level, and the time span represents the actual time length corresponding to a single coefficient at this scale. The length of the coefficient array of the macroscopic time series pattern is the length of the original time series feature component divided by the corresponding scale factor. The scale factors of the 3rd and 4th layers are 8 and 16, respectively.

[0045] The microcosmic time series pattern extraction process selects the 0th, 1st, and 2nd layer wavelet coefficients as short-term fluctuation features. These three layers of coefficients reflect the time scale changes from seconds to minutes. The microcosmic time series pattern uses the same data structure as the macroscopic time series pattern, but the length of the coefficient array is larger, containing more detailed information. The length of the 0th layer coefficient in the microcosmic time series pattern is equal to the length of the original time series feature component, and the lengths of the 1st and 2nd layer coefficients are half and one fourth of the original length, respectively.

[0046] The trend prediction unit adopts an autoregressive moving average model for trend extrapolation. The model parameters include two hyperparameters: the autoregressive order and the moving average order. The autoregressive order is set to 3, indicating that the data from the previous 3 time points are used to predict the value at the next time point. The moving average order is set to 2, indicating that the influence of the previous 2 prediction errors is considered. The input of the trend prediction unit is the coefficient array of the macro time series pattern, and the output is the baseline trend curve in the future time window. The length of the curve is determined according to the size of the prediction time window, and the default is to predict the trend change in the next 30 minutes.

[0047] The generation process of the baseline trend curve adopts a recursive prediction method. Using the model parameters trained from historical data, the predicted values at each future time point are calculated step by step. A sliding window is maintained during the prediction process, and the window size is equal to the autoregressive order. The window content is the true value or predicted value at the last few time points. After each prediction is completed, the new predicted value is added to the sliding window, and the oldest value is removed from the window, keeping the window size unchanged. The numerical range of the baseline trend curve is determined through statistical analysis of historical data, and the predicted values outside the reasonable range are truncated.

[0048] The volatility prediction unit adopts a Gaussian process regression model to process the micro time series pattern. The model uses a radial basis function as the covariance kernel function. The length scale parameter of the radial basis function is set to 0.5, which controls the correlation strength between adjacent time points. The noise variance parameter is set to 0.01, which reflects the random noise level in the observed data. The training data for the volatility prediction unit is the micro time series pattern within the historical time window, and the training process optimizes the kernel function parameters through maximum likelihood estimation.

[0049] The calculation process of the volatility offset inputs the micro time series pattern into the trained Gaussian process model to obtain the volatility prediction values and their uncertainty estimates at each time point in the future time window. The volatility offset uses the predicted mean as the point estimate, and the predicted variance is used for subsequent confidence calculation. The time resolution of the volatility offset is consistent with that of the micro time series pattern, usually with second or minute level accuracy. The length of the volatility offset sequence is the same as that of the baseline trend curve, ensuring that the two can be superimposed point by point.

[0050] The event arrival rate curve is generated by point-by-point superposition of the benchmark trend curve and the fluctuation offset, and the superposition operation is realized by numerical addition. The time alignment problem needs to be handled during the superposition process. Since the benchmark trend curve and the fluctuation offset have different time resolutions, the linear interpolation method is used to unify them to the same time grid. The numerical range of the event arrival rate curve is non-negative real numbers, and the negative value result is processed by zero value truncation. The event arrival rate curve contains two fields of timestamp array and arrival rate array. The timestamp array records each prediction time point, and the arrival rate array records the corresponding event arrival rate prediction value.

[0051] The calculation of the prediction stability is based on the stationarity analysis of the macro time series pattern. The stationarity is judged by the augmented Dickey-Fuller test. The calculation of the test statistic uses the coefficient sequence of the macro time series pattern. The null hypothesis of the test is that the sequence has a unit root, i.e., non-stationary. The alternative hypothesis is that the sequence is stationary. The critical value of the test is set to the standard value corresponding to the 5% significance level. When the test statistic is less than the critical value, the null hypothesis is rejected, and it is considered that the sequence is stationary. The prediction stability is defined as the complement of the p-value of the stationarity test. The smaller the p-value, the stronger the stationarity, and the higher the prediction stability.

[0052] The calculation of the prediction volatility is based on the randomness analysis of the micro time series pattern. The randomness is quantified by the entropy value of the sequence. The entropy value calculation processes the coefficient sequence of the micro time series pattern in segments, each segment containing 10 consecutive values. The probability distribution of the values in each segment is calculated, and then the Shannon entropy of the distribution is calculated. The prediction volatility is defined as the normalized entropy value, with a value range of 0 to 1. The larger the value, the stronger the randomness, and the higher the uncertainty of the prediction.

[0053] The prediction confidence is obtained by weighted fusion of the prediction stability and the prediction volatility. The fusion weight is optimized according to the historical prediction performance. The weight of the prediction stability is set to 0.6, and the weight of the prediction volatility is set to 0.4. The sum of the weights is 1 to ensure the normalization characteristics of the confidence. The calculation process of the prediction confidence is to multiply the prediction stability by its weight and subtract the prediction volatility multiplied by its weight. The result is mapped to a probability value between 0 and 1 by the sigmoid function. The closer the prediction confidence is to 1, the more reliable the prediction result is. The closer it is to 0, the less reliable the prediction result is.

[0054] The calculation of the load change rate uses the first-order difference method of the event arrival rate curve to calculate the change amplitude of the arrival rate between adjacent time points. The load change rate is defined as the arrival rate at the current time point minus the arrival rate at the previous time point, and then divided by the time interval to obtain the change rate. The load change rate can be positive or negative. A positive value indicates an increase in load, and a negative value indicates a decrease in load. The length of the load change rate sequence is one data point less than the event arrival rate curve, because the first time point cannot calculate the change rate.

[0055] The setting of the preset load threshold is based on statistical analysis of historical load data, and the threshold value is the 95th percentile of the historical load change rate distribution, ensuring that only 5% of the historical data exceeds the threshold. The preset load threshold is divided into two values, positive threshold and negative threshold, corresponding to the cases of sharp load rise and sharp load drop respectively. The positive threshold is set to 2.0 events per second per minute, and the negative threshold is set to -2.0 events per second per minute. The identification of the load change period is achieved by comparing the load change rate with the preset load threshold point by point, and the continuous time interval exceeding the threshold constitutes a load change period.

[0056] The extraction process of semantic type distribution is aimed at the identified load change period, and the historical event data in the period is traced back to count the occurrence frequency of various semantic types. The semantic type distribution is represented in the form of probability distribution, and the probability value of each semantic type is the occurrence frequency divided by the total number of events in the period. The data structure of semantic type distribution is a hash table, with the key as the semantic type identifier and the value as the corresponding probability value. The precision of the probability value is retained to 3 decimal places, and the sum of the probability values is equal to 1.

[0057] The calculation of resource consumption intensity is based on the resource demand label of each semantic type, which records the consumption intensity score of CPU, memory, network and storage. The comprehensive score of resource consumption intensity is calculated by weighted average of the intensity of the four resources, with CPU weight set to 0.4, memory weight set to 0.3, network weight set to 0.2, and storage weight set to 0.1. The comprehensive score ranges from 0 to 1, and the larger the value, the higher the resource consumption intensity.

[0058] The selection process of the dominant semantic type set sorts the semantic types in descending order according to the comprehensive score of resource consumption intensity, and selects the top 3 semantic types as the dominant semantic types. When the total number of semantic types is less than 3, all semantic types are selected as the dominant semantic types. The data structure of the dominant semantic type set is an ordered list, which contains three fields of semantic type identifier, probability value and resource consumption intensity.

[0059] The structured packaging of the comprehensive load descriptor is implemented in JSON format, and the JSON object contains five top-level fields. The event arrival rate curve field stores the timestamp array and arrival rate array, the prediction confidence field stores a single value, the load change period field stores the time identifier array, the dominant semantic type set field stores the semantic type object array, and the metadata field stores the generation time and version information. The serialization of the JSON object uses compact format, removing unnecessary white space, and compressing the storage space.

[0060] With a specific data case to illustrate the above implementation process, assume that the input event multi-dimensional feature vector is a 128-dimensional array, the first 64 dimensions are time series feature components, and the last 64 dimensions are semantic feature components. The time series feature components are decomposed by 4 layers of wavelet, obtaining 8 coefficients of the 3rd layer and 4 coefficients of the 4th layer as macro time series patterns, 64 coefficients of the 0th layer, 32 coefficients of the 1st layer and 16 coefficients of the 2nd layer as micro time series patterns. The trend prediction unit predicts the baseline trend curve of the future 30 minutes based on 12 macro coefficients, containing 30 data points, the value range is 0.8 to 3.2 events per second. The fluctuation prediction unit predicts the fluctuation offset based on 112 micro coefficients, the value range is -0.5 to 0.5 events per second. The superimposed event arrival rate curve value range is 0.3 to 3.7 events per second. The stationary test p value of the macro time series pattern is 0.03, the prediction stability is 0.97, the entropy value of the micro time series pattern is 0.85, the prediction fluctuation is 0.85, and the fusion prediction confidence is 0.24. The load change rate calculation finds that the time period from 15 to 20 minutes exceeds the positive threshold value 2.0, the data query event proportion in this period is 60%, the file upload event proportion is 30%, the user login event proportion is 10%, and the resource consumption intensity is 0.7, 0.9 and 0.3 respectively, and the dominant semantic type set is file upload and data query. The final generated comprehensive load descriptor JSON object size is 2.1KB, containing complete prediction results and semantic analysis information.

[0061] In an alternative embodiment, the macro time series pattern is input into the trend prediction unit to obtain the baseline trend curve, and the micro time series pattern is input into the fluctuation prediction unit to obtain the fluctuation offset, and superimposing the baseline trend curve and the fluctuation offset to generate the event arrival rate curve comprises: A fractal dimension mapping is established for the macro time series pattern, the complexity characteristics of the monotone increasing and decreasing trend segments in the macro time series pattern are quantified, a fractal predictor is constructed based on the complexity characteristics, the fractal predictor is evolved and deduced in a future time window to obtain a trend extension sequence corresponding to each trend segment, the connection characteristics of the trend segment boundaries are optimized by the fractal predictor to generate a continuous baseline trend curve; The micro time series pattern is subjected to double feature decoupling, the micro time series pattern is projected into a periodicity and randomness feature space, the fluctuation period and fluctuation amplitude are quantified in the periodicity feature space, a fractal predictor is constructed to reconstruct the periodic fluctuation characteristics, the reconstructed periodic fluctuation characteristics are iteratively evolved in a future time window to obtain a periodic fluctuation sequence, a fluctuation intensity description is established for the randomness feature space to generate a random fluctuation sequence, and the periodic fluctuation sequence and the random fluctuation sequence are adaptively combined to obtain a fluctuation offset; The baseline trend curve is feature-fused with the fluctuation offset in the time dimension to generate an event arrival rate curve containing trend evolution information and fluctuation characteristic information.

[0062] The fractal dimension mapping of macro time series is realized by box counting algorithm, which embeds time series data into two-dimensional coordinate system, with time index as horizontal axis and numerical value as vertical axis. The algorithm uses a square grid with side length ε to cover the data curve, with ε values from 0.05 to 5.0, with 8 scales set by 2 times increment. For each scale, the number of grids containing the curve N(ε) is counted, and the linear regression slope of log(N(ε)) and log(1 / ε) is calculated as the fractal dimension D. The fractal dimension calculation module maintains the scale array and grid count array, and fits the slope of the straight line by least squares method. The absolute value of the slope is the fractal dimension estimate, usually ranging from 1.2 to 1.8.

[0063] The monotone increasing and decreasing trend segment identification uses a sliding window difference detection method, with window length set to 7 data points and step length of 1. The slope k of the first and last points in the window is calculated, and when k is greater than the threshold value 0.08, it is marked as increasing, and when it is less than-0.08, it is marked as decreasing, otherwise it is stable. Adjacent windows of the same type are merged into a trend segment, and the number of consecutive windows is required to be not less than 3 when merging. The trend segment records four attributes: starting position, ending position, segment length and average slope, with segment length usually ranging from 5 to 25 data points.

[0064] The complexity feature quantification combines fractal dimension and trend segment geometric characteristics, with the formula being complexity C equal to fractal dimension D multiplied by the square root of segment length L multiplied by adjustment coefficient α. The α of increasing segment is 1.15, the α of decreasing segment is 0.92, and the α of stable segment is 1.0. The complexity value is normalized to map to the range of 0.6 to 2.8. The complexity calculation module outputs a complexity array, with array length equal to the number of trend segments, and each element corresponding to the complexity value of a trend segment.

[0065] The fractal predictor constructs a parameter mapping table based on complexity features, containing three parameters: extrapolation strength coefficient β, time decay coefficient γ and noise amplitude coefficient δ. β is mapped by piecewise linear function, with complexity 0.6 corresponding to β 1.8, complexity 2.8 corresponding to β 0.7, and intermediate values linearly interpolated. The mapping range of γ is 0.015 to 0.085, and the mapping range of δ is 0.008 to 0.045. The predictor parameter calculation module receives the complexity array and outputs the corresponding parameter array, with parameter precision retained to 3 decimal places.

[0066] The trend extrapolation sequence evolution uses an autoregressive prediction model to calculate the extrapolation value of the next 30 steps for each trend segment. The t-step extrapolation value is equal to the previous step value plus the segment slope multiplied by β multiplied by exp(-γ×t) plus a Gaussian noise term N(0,δ 2). The evolution process maintains a state vector, recording the current continuation value, accumulated time, and error accumulation. The evolution calculation is performed in a GPU parallel environment, and the 30-step continuation of a single trend segment takes about 0.3 milliseconds.

[0067] The trend segment boundary connection is optimized by cubic spline interpolation, and a transition interval of 5 time steps is set at the boundary of adjacent segments. The values in the transition interval are calculated by weighted combination of the continuation values of the adjacent two segments, and the weight function w(t) is equal to 0.5 times (1+cos(πt / 5)), t is the relative time in the transition interval. The connection optimization module checks the boundary continuity, ensures the continuity of the first derivative, and the smoothness of the second derivative. The numerical error after connection is controlled within 5% of the original data standard deviation.

[0068] The continuous reference trend curve generation splices the continuation sequences of each segment along the time axis to generate a sequence of binary tuples containing timestamps and trend values. The splicing process handles the time overlap problem, and the overlapping area takes the weighted average value, and the weight is proportional to the segment length. The reference curve data structure contains a time array and a numerical array with a length of 30, and the array index corresponds to the future minute number. The numerical range is constrained within a reasonable interval by historical extreme values. The curve smoothness is tested by second-order difference, and abnormal jump points are corrected by median filtering.

[0069] The micro-timing pattern dual feature decoupling uses the ensemble empirical mode decomposition method to superimpose 6 different white noise sequences on the original signal, and the noise amplitude is 0.2 times the standard deviation of the signal. The decomposition process produces 8 intrinsic mode functions and 1 residual term, the first 4 IMFs are classified as periodic components, and the last 4 IMFs and the residual term are classified as random components. The termination condition of the decomposition iteration is that the correlation coefficient of the results of the last two screenings is greater than 0.98 or the number of iterations exceeds 50.

[0070] Periodic feature space projection extracts the instantaneous frequency and instantaneous amplitude information of the first 4 IMFs through Hilbert transform. The feature space is constructed as a 4-dimensional vector, each dimension corresponds to the energy proportion of an IMF. The energy is calculated as the sum of the squares of the IMF sequence divided by the sum of the squares of the total signal, and the sum of the energy proportions of the 4 dimensions is about 0.7 to 0.9. The projected feature vector is reduced to 3 dimensions through principal component analysis, retaining 95% of the information.

[0071] Randomness feature space projection calculates the statistical features of the last 4 IMFs and the residual term, including variance σ 2 , third central moment μ3 and fourth central moment μ4. Feature standardization maps the variance to the 0 to 1 interval, and the skewness and kurtosis are standardized by Z-score. The randomness feature vector is 3-dimensional, corresponding to the standardized variance, skewness and kurtosis values respectively. Feature calculation uses a sliding window strategy with a window length of 20 data points and an overlap rate of 50%.

[0072] The period and amplitude quantification of the fluctuation is achieved by power spectral density analysis, and 512-point fast Fourier transform is performed on each periodic IMF. The period identification searches for the power spectrum peak in the frequency domain, and the reciprocal of the frequency corresponding to the peak is the fluctuation period. The amplitude quantification is the square root of the power spectral density at the peak frequency multiplied by the correction factor 2.0. The typical fluctuation period distribution is four frequency bands of 2 to 3 minutes, 5 to 8 minutes, 12 to 18 minutes, and 25 to 35 minutes.

[0073] The fractal predictor reconstructs the periodic fluctuation using phase space reconstruction technology, and the embedding dimension is set to 4 and the delay time is set to 2 sampling points. The reconstructed trajectory evolves in a 4-dimensional phase space, and the next step position of the trajectory is predicted by the k-nearest neighbor method. The number of neighbors k is set to 8, and the similarity measure uses weighted Euclidean distance, and the weight vector is [1.0, 0.8, 0.6, 0.4]. The prediction accuracy is evaluated by cross-validation, and the average absolute error is controlled within 15% of the original fluctuation amplitude.

[0074] The periodic fluctuation sequence iterative evolution performs 30-step forward prediction based on the phase space trajectory, and each step prediction uses 8 nearest neighbors of the current state. The predicted value is the distance weighted average of the next step state of the nearest neighbor points, and the weight function is exp(-d 2 / 2σ 2 ), d is the Euclidean distance, and σ is the neighborhood standard deviation. The evolution process introduces a forgetting factor of 0.05 to reduce the weight of long-term prediction and improve the stability of prediction.

[0075] The fluctuation intensity description is based on time series analysis of randomness characteristics, and the GARCH model is used to describe the dynamic evolution of variance. The model parameters α and β are determined by maximum likelihood estimation, and α controls the short-term fluctuation influence and β controls the long-term persistence. The fluctuation intensity prediction uses the recursive formula, and the current intensity is equal to the long-term variance multiplied by α multiplied by the square of the previous residual plus β multiplied by the previous intensity. The intensity prediction error is evaluated by Monte Carlo simulation, and the coverage rate of the confidence interval reaches 90%.

[0076] The random fluctuation sequence generation uses a conditional heteroscedasticity model to generate random disturbance items based on the predicted fluctuation intensity. The random number generator uses the Mersenne Twister algorithm, and the seed value is set based on the system timestamp. The generated random sequence is verified for distribution fitting by Kolmogorov-Smirnov test, and the p-value is required to be greater than 0.05. The sequence length is 30 data points, consistent with the periodic fluctuation sequence.

[0077] The adaptive combination strategy dynamically adjusts the weights of periodic and random components, and the weight calculation is based on the signal-to-noise ratio of the two types of components. The periodic weight w1 is equal to the periodic signal power divided by the total power, and the random weight w2 is equal to 1 minus w1. The combined fluctuation offset is w1 times the periodic fluctuation sequence plus w2 times the random fluctuation sequence. The weight update frequency is once every 10 data points, ensuring that the combination result can adapt to changes in the timing characteristics.

[0078] Feature fusion is a point-by-point addition operation that time-aligns the reference trend curve with the fluctuation offset. Before fusion, length check and timestamp calibration are performed on the two sequences. The fusion result is smoothed by a 5-point moving average filter to smooth high-frequency noise, and the filter coefficients are [0.1, 0.2, 0.4, 0.2, 0.1]. The final event arrival rate curve guarantees the non-negativity constraint, and negative values are set to the minimum value of 0.01 through truncation processing.

[0079] The data case verification uses a historical time series containing 96 data points with a value range of 0.8 to 6.2. The macro pattern identifies 3 trend segments, with a fractal dimension of 1.52 and a complexity feature of [1.8, 2.1, 1.6]. The micro pattern decomposition produces fluctuation periods [3.2, 7.5, 14.8, 28.3] minutes, corresponding to amplitudes [0.25, 0.18, 0.12, 0.08]. The generated event arrival rate curve exhibits obvious trend and periodic fluctuation characteristics within a 30-minute prediction window, with a value range of 1.2 to 5.8, and the curve continuity and smoothness meet the requirements of practical applications.

[0080] In an optional implementation, the load peak period is quantified according to the comprehensive load descriptor, the processing nodes in the resource topology are grouped in combination with the semantic processing requirements corresponding to the load peak period, and the number of nodes and routing weights are reconstructed according to the gradient distribution of the event arrival rate curve to generate a dynamic resource topology structure, including: An entropy value mapping is established for the event arrival rate curve in the comprehensive load descriptor to quantify the time interval characteristics of the event arrival rate exceeding the historical average, the integral load quantity in each time interval is calculated, and the load peak period is determined based on the integral load quantity; The dominant semantic type set associated with the load peak period is mapped to the entropy value space, a matching entropy metric is constructed in combination with the semantic ability labels of the processing nodes in the resource topology, the organization structure of the processing nodes is reconstructed according to the matching entropy metric, and a matching node group is generated; An entropy value mapping is established for the event arrival rate curve to obtain a gradient distribution curve, the characteristics of the load sudden change moments in the gradient distribution curve are quantified, the resource demand increment is deduced based on the event arrival rate growth trend before the load sudden change moments, and the resource demand increment is converted into the node expansion quantity of each matching node group; According to the node expansion quantity, resource distribution of each matching node group is reconstructed, load saturation of processing nodes in the matching node group is quantified, routing weight is constructed based on reciprocal of the load saturation, and dynamic resource topology structure is generated by topological coding of organization structure, node information, quantity distribution and routing weight of the matching node group.

[0081] The entropy value mapping of the event arrival rate curve in the comprehensive load descriptor is realized by using an information entropy calculation method. The event arrival rate numerical sequence is divided into 20 equal-width intervals, and the frequency of data points in each interval is counted. The entropy value calculation module takes the logarithm of the frequency of each interval and sums it up to obtain the information entropy value of the event arrival rate distribution. The entropy value range is usually between 2.5 and 4.2, and the larger the value, the more uniform the event arrival rate distribution, and the smaller the value, the more concentrated the distribution. The entropy value mapping process maintains a sliding window with a length of 15 time points, and the window slides by 2 time points. The local entropy value is calculated for each window position to form an entropy value time sequence.

[0082] The calculation of the historical average is based on the event arrival rate data of the past 30 days, and uses a weighted average method. The weight of recent data is set to 1.0, and the weight decays exponentially over time with a decay coefficient of 0.95. The identification of time intervals exceeding the historical average is realized by threshold comparison. When the event arrival rate is greater than the threshold coefficient of 1.25 times the historical average, the time point is marked as a peak candidate point. Continuous peak candidate points are merged into peak intervals, and the merging condition requires that the interval between consecutive time points does not exceed 3 minutes, and the minimum duration of the peak interval is set to 5 minutes.

[0083] The time interval feature quantization process calculates three characteristic parameters of the duration, peak amplitude and change rate of each peak interval. The duration is the end time minus the start time, the peak amplitude is the difference between the maximum event arrival rate in the interval and the historical average, and the change rate is the peak amplitude divided by the duration. The characteristic parameters are normalized, the duration is mapped to the range of 0 to 1, and the peak amplitude and change rate are standardized by Z-score.

[0084] The integral load quantity is calculated by numerically integrating the event arrival rate curve in each peak interval, using the trapezoidal integration method. The interval is divided into several small trapezoids, and the area of each trapezoid is the average of the upper and lower base multiplied by the height. The integral load quantity represents the total load accumulation in the peak interval, and the numerical unit is event number multiplied by time. The integral calculation precision is controlled within one thousandth of the original data precision, and the outliers are smoothed by the median filtering method.

[0085] The determination of the load peak period is based on the sorting of the integral load and threshold screening. All peak intervals are sorted in descending order of integral load, and the top 70% intervals are selected as the load peak period. The peak period determination module outputs a peak period list, each peak period recording the start time, end time, integral load and characteristic parameters. The average duration of the peak period is about 12 to 18 minutes, and the numerical range of the integral load is dynamically adjusted according to the specific application scenario.

[0086] The mapping process of the dominant semantic type set analyzes the semantic task type distribution processed in each load peak period, and calculates the proportion of tasks of semantic types such as text processing, image recognition, speech analysis and data mining. The dominant semantic type is defined as a semantic type with a proportion of more than 30%, and each peak period contains 1 to 3 dominant semantic types. The semantic type set is represented by a vector, and the vector dimension is equal to the total number of semantic types. The value of each dimension is the normalized proportion of the corresponding semantic type.

[0087] The construction of the entropy value space maps the dominant semantic type set to a high-dimensional entropy space, and the dimension number of the entropy space is equal to the total number of semantic types. Each dimension corresponds to the entropy value of a semantic type. The calculation of the semantic type entropy value is based on the complexity distribution of the type task, which is divided into three levels of simple, medium and complex, and is classified by task execution time and resource consumption. The entropy value is calculated using the Shannon entropy formula, and the information entropy of the task number distribution of the three complexity levels is calculated.

[0088] The semantic ability label of the processing node contains three attributes of semantic type supported by the node, processing capacity level and specialization degree. The semantic type support is represented by a binary vector, and the vector length is equal to the total number of semantic types. The supported type corresponds to 1 at the corresponding position, and the unsupported position is 0. The processing capacity level is divided into three levels of low, medium and high, corresponding to numerical coding of 1, 2 and 3 respectively. The specialization degree is calculated by the historical processing efficiency of the node in a specific semantic type, and the efficiency value ranges from 0.5 to 2.0.

[0089] The construction of the matching entropy measure uses weighted Euclidean distance to calculate the similarity between the dominant semantic type set and the semantic ability of the processing node. The distance calculation process calculates the square of the difference between the set entropy value and the node ability for each semantic type dimension, multiplies it by the weight coefficient of the semantic type, and the weight coefficient is determined according to the importance of the semantic type in the current load. The numerical range of the matching entropy measure is 0 to 10, and the smaller the value, the higher the matching degree.

[0090] The reconstruction of the processing node organization structure is based on clustering analysis of the matching entropy metric, and the k-means clustering algorithm is used to divide the processing nodes into several matching node groups. The clustering number k is determined according to the number of dominant semantic types, and is usually set to 1.5 times the number of dominant semantic types or more. The clustering process iteratively optimizes node grouping to minimize the variance of the matching entropy metric of nodes within the group and maximize the difference of the matching entropy metric of nodes between groups. The clustering convergence condition is that the variance within the group changes by less than 0.01 for 5 consecutive iterations.

[0091] The generation result of the matching node group contains group identification, node list within the group, semantic feature of the group, and average matching entropy metric of the group. The semantic feature of the group is calculated by weighted average of the semantic ability labels of the nodes within the group, and the weight is the processing capacity level of the node. Each matching node group maintains group state information, including current load level, available resource quantity and response time statistics. The minimum number of nodes in the group is set to 3, and the maximum number of nodes is set to 50.

[0092] The construction of the gradient distribution of the event arrival rate curve calculates the event arrival rate change rate between adjacent time points by numerical difference method. The gradient calculation uses the central difference format, and the gradient at the current time point is equal to the value at the next time point minus the value at the previous time point divided by 2 times the time interval. The positive and negative of the gradient value represents the increasing and decreasing trend of the event arrival rate, and the absolute value size represents the degree of change. The gradient distribution curve has the same time resolution and length as the original event arrival rate curve.

[0093] The entropy value mapping process of the gradient distribution curve divides the gradient value into three categories: positive gradient, zero gradient and negative gradient, corresponding to the rising, stable and falling stages of the event arrival rate. The entropy value is calculated based on the time proportion distribution of the three categories, and the ternary entropy formula is used to calculate the information entropy. The time series of gradient entropy values is generated by the sliding window method, with a window length of 10 time points and an overlap rate of 50%.

[0094] The feature quantization of the load sudden change time is realized by detecting the mutation of the gradient absolute value, and when the gradient absolute value exceeds 2.5 times the historical gradient standard deviation, it is marked as a sudden change candidate time. The sudden change time confirmation needs to meet two conditions: local maximum of gradient absolute value and consistency of gradient sign within 5 minutes before and after. The sudden change time features include sudden change intensity, sudden change direction and sudden change duration, and the sudden change intensity is the ratio of the gradient absolute value to the historical standard deviation.

[0095] The event arrival rate growth trend is derived based on the gradient change trend in the 10 minutes before the critical moment. The linear regression method is used to fit the time evolution pattern of the gradient. The slope parameter of the regression model represents the acceleration of the gradient change, and the intercept parameter represents the initial gradient value. The growth trend prediction is obtained by extrapolating the gradient change of the next 5 minutes through the regression model, and the expected increment of the event arrival rate is obtained by integration. The prediction confidence is evaluated by the determination coefficient of the regression model, which requires a determination coefficient greater than 0.6.

[0096] The resource demand increment is converted from the expected increment of the event arrival rate to the demand of processing resources, and the conversion coefficient is determined based on the statistical relationship between historical load and resource consumption. The conversion coefficient is calculated using the sliding average method, using the data of the past 20 critical events for statistics, and the numerical range of the conversion coefficient is usually between 1.2 and 3.8. The resource demand increment includes three dimensions of computing resources, storage resources and network resources, and the demand of each dimension is allocated according to the resource consumption characteristics of the semantic task type.

[0097] The node expansion quantity is determined by allocating the resource demand increment according to the semantic characteristics of the matching node group, and the allocation weight is calculated based on the matching degree of the dominant semantic type of each group in the current load peak period. The node expansion quantity is equal to the resource demand increment allocated to the group divided by the average processing capacity of a single node, and the result is rounded up to ensure sufficient resources. The expansion quantity calculation considers the node startup time and resource scheduling delay, and the advance amount is set to 3 minutes, and the upper limit of the expansion quantity is 2 times the current number of nodes in each group.

[0098] The resource distribution reconstruction process of the matching node group adds the newly expanded nodes to the corresponding node group, updates the node list and resource statistical information of the group. The resource distribution reconstruction maintains the load balancing constraint, and distributes the new load evenly to each node in the group through the load dispersion strategy. The reconstruction process monitors the node resource utilization rate, and triggers further expansion when the utilization rate exceeds 85%, and triggers resource recycling when the utilization rate is less than 30%.

[0099] The load saturation degree is quantitatively calculated as the ratio of the current load of each processing node to the maximum processing capacity, and the load saturation degree ranges from 0 to 1, 0 represents idle node, and 1 represents full load operation. The load saturation degree is calculated based on the weighted average of the CPU usage rate, memory occupancy rate and task queue length of the node, and the weight coefficients are 0.4, 0.3 and 0.3 respectively. The saturation degree calculation precision is retained to 2 decimal places, and the update frequency is every 30 seconds.

[0100] The construction of routing weight is based on the inverse relationship of load saturation, the routing weight is equal to 1 divided by the load saturation plus a smoothing term of 0.1, the smoothing term avoids division by zero error and provides a basic weight. The weight value ranges from 1.0 to 10.0, the larger the weight, the lighter the node load, and the node is preferentially selected for task allocation. The routing weight is normalized to ensure that the sum of the weights of all nodes in the group is equal to 1, and the normalized weight is used as a probability distribution for random routing selection.

[0101] The topology encoding process integrates the organizational structure, node information, quantity distribution and routing weight information of the matching node group into a structured topology description file. The encoding format uses JSON format, which includes four main fields: group list, node list, connection relationship and weight matrix. The group list records the identification, semantic length and node quantity of each group, and the node list records the identification, group, capability label and current state of each node. The connection relationship describes the communication link between nodes, and the weight matrix records the routing weight distribution.

[0102] The generation of dynamic resource topology structure deploys the encoded topology information to the distributed scheduler, and the scheduler allocates tasks and balances loads according to the topology structure information. The topology structure supports a hot update mechanism, and when the node expands or the load changes, the scheduler updates the topology information in real time without interrupting the running tasks. The validity period of the topology structure is set to 30 minutes, and it is regenerated after timeout to adapt to the load changes.

[0103] The data case demonstrates the entire implementation process, assuming that the input event arrival rate curve contains 120 data points, the historical average is 45 events per minute, and 4 load peak periods are identified, with integral load quantities of 1250, 980, 1150 and 1320 respectively. The dominant semantic types include text processing and image recognition, with entropy values of 2.8 and 3.1 respectively. There are 32 existing processing nodes, which are divided into 3 matching node groups after matching entropy measurement, with node quantities of 12, 10 and 10 respectively. The gradient distribution curve shows 2 load change moments, with resource demand increments of 280 processing units, and node expansion quantities of 4, 3 and 3 respectively. The reconstructed load saturation range is 0.35 to 0.78, and the routing weight distribution is [0.42, 0.31, 0.27]. The generated dynamic resource topology structure contains 35 nodes distributed in 3 groups, supporting real-time task scheduling and load balancing.

[0104] In an alternative embodiment, the node distribution, routing weight of the dynamic resource topology structure and the prediction confidence are mapped to the optimization space, a constrained problem is constructed with resource utilization and processing delay as optimization objectives, the resource redundancy threshold is evolved based on the prediction confidence, and a scaling decision scheme is generated, including: The node position identifier and connection relationship in the dynamic resource topology structure are constructed into a spectral clustering space, the node distribution is mapped into a distribution feature vector in the spectral clustering space, an association matrix is constructed based on a routing weight, and the prediction confidence is embedded into the association matrix to obtain a space modulation parameter; A dual-objective optimization architecture is constructed in the spectral clustering space, resource utilization is represented by distribution uniformity of the distribution feature vector, processing delay is represented by distribution characteristics of the association matrix, a comprehensive optimization target is obtained by spectral clustering combination of the dual objectives under the space modulation parameter, and constraint conditions are set for the distribution feature vector and the association matrix in the dual-objective optimization architecture; A spectral clustering compensation term is constructed based on the prediction confidence, the spectral clustering compensation term is applied to the comprehensive optimization target to generate a resource redundancy threshold, and the constraint boundary of the dual-objective optimization architecture is adjusted according to the resource redundancy threshold; An optimal resource configuration scheme is solved in the spectral clustering space, the optimal resource configuration scheme is compared with the current node distribution, the number of nodes to be expanded or contracted is determined based on the comprehensive optimization target, the optimal resource configuration scheme is converted into a routing weight distribution ratio according to the space modulation parameter, and an expansion and contraction decision scheme is generated.

[0105] The node position identifier in the dynamic resource topology structure is represented by a two-dimensional coordinate system, each node is assigned a unique node identifier and a corresponding coordinate position. The node position coordinates are calculated by a load balancing algorithm, the horizontal coordinate represents the processing capacity level of the node, and the value range is 1 to 10, and the vertical coordinate represents the semantic specialization degree of the node, and the value range is 0.1 to 1.0. The coordinate calculation module receives historical processing data and capacity parameters of the node, and calculates the coordinate values by a weighted average method. The processing capacity level is calculated based on three indexes of core number, memory capacity and historical throughput of the node, and the weight coefficients are 0.4, 0.3 and 0.3 respectively. The semantic specialization degree is calculated by the ratio of the processing efficiency of the node on a specific semantic type to the average efficiency, and the ratio greater than 1 indicates high specialization degree.

[0106] The construction of the connection relationship is represented by an adjacency matrix, whose elements are 0 or 1. A value of 1 indicates that there is a direct communication link between two nodes, and a value of 0 indicates that there is no connection. The connection relationship establishment module measures the network delay and bandwidth capacity between nodes. When the delay is less than 5 milliseconds and the bandwidth is greater than 100 megabits per second, the connection is marked as 1 in the adjacency matrix. The delay measurement is obtained by averaging the results of 10 tests using a network probe command, and the bandwidth measurement is obtained by testing for 30 seconds using a bandwidth test tool to obtain a stable bandwidth value. The connection relationship matrix is an n by n symmetric matrix, where n is the total number of nodes. The diagonal elements of the matrix are set to 0 to indicate that a node is not connected to itself. The adjacency matrix construction process includes connectivity checking to ensure the connectivity of the graph, and isolated nodes are connected to the nearest neighbor node by force.

[0107] The construction of the spectral clustering space is based on the eigenvalue decomposition process of the Laplacian matrix, which is equal to the degree matrix minus the adjacency matrix. The degree matrix is a diagonal matrix, and the diagonal elements are equal to the sum of the elements in the corresponding row of the adjacency matrix, representing the connection degree of the node. The eigenvalue decomposition of the Laplacian matrix uses the Jacobi method for iterative calculation, and the iteration termination condition is that the change of the eigenvalue of the last two iterations is less than 0.000001. The eigenvalues are sorted in ascending order, and the eigenvectors corresponding to the first k smallest non-zero eigenvalues are selected to form the basis vectors of the spectral clustering space, and the value of k is set to the square root of the total number of nodes n, rounded up. The eigenvectors are orthogonalized to ensure that the basis vectors are mutually orthogonal.

[0108] The mapping process of the node distribution feature vector projects the node position in the two-dimensional coordinate space into the k-dimensional spectral clustering space. The projection calculation uses matrix multiplication operation, and the node coordinate matrix is an n by 2 matrix, and the eigenvector matrix is an n by k matrix. The distribution feature vector is equal to the transpose of the coordinate matrix multiplied by the eigenvector matrix. The projection weight is adjusted by the node importance, which is equal to the node connection degree multiplied by 0.6 plus the node processing capacity level multiplied by 0.4. The adjusted distribution feature vector element is equal to the original projection value multiplied by the importance weight of the corresponding node, and the dimension components are normalized by dividing by the component sum to ensure that the component sum is equal to 1. The distribution feature vector calculation module maintains historical statistical information for anomaly detection and trend analysis.

[0109] The construction of the routing weight correlation matrix combines the routing weight information with the node connection relationship. The element of the correlation matrix is equal to the corresponding element of the adjacency matrix multiplied by the routing weight value. The routing weight value is calculated based on the inverse of the node load saturation. When the node load saturation is s, the routing weight is equal to 1 divided by s plus a smoothing term of 0.1. The correlation matrix is an n by n symmetric matrix, and the matrix element ranges from 0 to 10. The correlation matrix construction process includes a weight normalization step. Each row element is divided by the sum of the elements in the row to ensure that the row sum is 1. The normalized correlation matrix is used for subsequent spectral analysis calculations. The weight calculation module supports multiple load indicators, including processor usage, memory occupancy, and network traffic, with weight coefficients of 0.5, 0.3, and 0.2, respectively.

[0110] The prediction confidence embedding process uses the confidence value as a spatial modulation parameter to perform element-level modulation on the correlation matrix. The spatial modulation parameter is equal to the square root of the prediction confidence, and the modulation parameter ranges from 0.6 to 1.0. The modulation operation performs power operation on each non-zero element of the correlation matrix. The modulated element value is equal to the original element value raised to the power of the modulation parameter. When the confidence is higher, the modulation parameter is closer to 1, and the modulation effect is smaller. When the confidence is lower, the modulation parameter is smaller, and the modulation effect is larger. The modulated correlation matrix is re-normalized by row to ensure that the randomness of the matrix remains unchanged. The modulation parameter calculation module records the modulation history for parameter stability analysis and abnormal modulation detection.

[0111] The dual-objective optimization architecture constructs two objective functions, resource utilization and processing delay, in the k-dimensional spectral clustering space. The resource utilization objective function is quantified by the distribution uniformity of the distribution feature vector, and the weighted variance of each dimension component is calculated. The weighted variance calculation formula is the square of each dimension component minus the mean multiplied by the sum of the weights, and the weight is equal to the inverse of the corresponding eigenvalue normalized. The more uniform the distribution, the smaller the variance, and the resource utilization objective function takes the inverse of the variance to make the uniform distribution correspond to a higher objective value. The uniformity calculation uses a sliding window to smooth out transient fluctuations, with a window length of 20 historical data points.

[0112] The processing delay objective function is represented by the distribution characteristics of the modulated correlation matrix, including the condition number and the spectral radius. The condition number is equal to the maximum eigenvalue of the correlation matrix divided by the minimum positive eigenvalue, and the spectral radius is equal to the absolute value of the maximum eigenvalue. The processing delay objective function is equal to the inverse of the condition number multiplied by 0.6 plus the inverse of the spectral radius multiplied by 0.4, and the weight coefficients 0.6 and 0.4 ensure that the condition number has a greater impact. The matrix eigenvalue calculation uses the power iteration method, and the iteration number is set to 100 to ensure convergence accuracy. The eigenvalue calculation module supports parallel computing to accelerate the feature decomposition process of large-scale matrices.

[0113] The comprehensive optimization objective function adopts a weighted linear combination method, and the comprehensive objective is equal to the resource utilization rate objective multiplied by a weight coefficient one plus the processing time delay objective multiplied by a weight coefficient two. The weight coefficient one and the weight coefficient two satisfy a constraint condition that both are greater than 0 and their sum is equal to 1. The default value of the weight coefficient one is set to 0.6 to reflect the importance of the resource utilization rate, and the default value of the weight coefficient two is set to 0.4. The weight coefficients support dynamic adjustment, and the adjustment trigger condition is that the current resource utilization rate is less than 70 percent to increase the value of the weight coefficient one, and the average processing time delay exceeds a threshold value to increase the value of the weight coefficient two. The dynamic weight adjustment adopts an exponential smoothing method, and the smoothing coefficient is 0.3 to ensure the stability of the weight change.

[0114] The constraint condition setting includes two types of distribution feature vector constraints and correlation matrix constraints. The distribution feature vector constraints include non-negativity constraints that each component is greater than or equal to 0 and normalization constraints that each dimension component sum is equal to 1. The correlation matrix constraints include symmetry constraints that the matrix is equal to the transpose of the matrix, non-negativity constraints that each element is greater than or equal to 0, and row sum constraints that each row element sum is less than or equal to the maximum processing capacity of the node. The maximum processing capacity of the node is determined by the node hardware configuration and historical peak load, and the calculation formula is the number of processor cores multiplied by 2 plus the memory capacity number divided by 4 and rounded up. The constraint checking module verifies the constraint condition in each optimization iteration, and uses a projection method to project the solution vector into the feasible region when the constraint is violated.

[0115] The spectral clustering compensation term is constructed based on the uncertainty quantification of the prediction confidence, and the compensation term reflects the influence of the confidence deviation on the optimization objective. The confidence deviation is equal to the current prediction confidence minus the historical average confidence, and the historical average confidence is calculated by the arithmetic average of the past 20 predictions through a sliding window. The spectral clustering compensation term is equal to the square of the confidence deviation multiplied by a compensation coefficient, and the default value of the compensation coefficient is 0.05, and the value range is 0.01 to 0.1. The compensation term calculation module recalculates the compensation term value after each prediction update. The adaptive adjustment of the compensation coefficient is based on the historical statistics of the prediction error, and the compensation coefficient is increased when the error is large, and the compensation coefficient is reduced when the error is small.

[0116] The resource redundancy threshold generation process adds the spectral clustering compensation term to the comprehensive optimization objective, and the modified optimization objective is equal to the original optimization objective plus the spectral clustering compensation term. The resource redundancy threshold is equal to the inverse of the modified optimization objective multiplied by an adjustment factor. The value range of the adjustment factor is 1.2 to 2.0, and the default value is 1.5. When the prediction model stability is higher, a smaller value is taken, and when the stability is lower, a larger value is taken. The resource redundancy threshold represents the additional resource proportion required to ensure the service quality under the prediction uncertainty, and the threshold calculation precision is retained to 2 decimal places. The threshold generation module maintains a historical sequence of the threshold for trend analysis and anomaly detection.

[0117] The constraint boundary adjustment extends the optimization architecture constraints based on the resource redundancy threshold, allowing a larger resource configuration search space. The node number upper bound constraint is extended from the original value to the original value multiplied by the resource redundancy threshold, and the connection bandwidth lower bound constraint is adjusted from the original value to the original value divided by the resource redundancy threshold. The upper bound of the processing capacity constraint is adjusted from the maximum value to the maximum value multiplied by the resource redundancy threshold, ensuring that the performance requirements can still be met under the prediction uncertainty. The constraint adjustment process checks for constraint consistency and reverts to the original constraints and records a warning message when the adjusted constraints conflict. The boundary adjustment uses a gradual method, with each adjustment not exceeding 20 percent of the original boundary, avoiding stability problems caused by drastic changes.

[0118] The optimal resource configuration solution uses a sequential quadratic programming algorithm to search for the optimal solution in the k-dimensional spectral clustering space. In the initialization stage, the initial solution vector is set to the current distribution feature vector, and the initial step size is 0.001. In the iteration process, the gradient of the objective function and the Jacobian matrix of the constraint function are calculated, and the search direction is determined by solving the quadratic programming subproblem. The golden section method is used to determine the optimal step size in the line search, with a step size search precision of 0.001. The algorithm convergence condition is that the change in the objective function value of the last two iterations is less than 0.001, or the gradient norm is less than 0.001, or the number of iterations exceeds 200. The optimization solving module supports a multi-start point strategy, starting optimization from different initial solutions to avoid local optimal solutions.

[0119] The difference comparison calculation uses the weighted Euclidean distance to measure the difference between the optimal configuration and the current configuration. The distance calculation formula is the square root of the sum of the square of the difference in each dimension multiplied by the dimension weight. The dimension weight is equal to the ratio of the corresponding feature value to the sum of all feature values, ensuring that important dimensions receive higher weights. The difference comparison module outputs three lists: the expanded node list contains the node identifiers and the amount of resources to be added, the reduced node list contains the node identifiers and the amount of resources to be reduced, and the weight adjustment list contains the connection pairs and new weight values that need to be modified. The difference threshold is set to 0.15, and when the difference in a certain dimension exceeds this threshold, the corresponding adjustment operation is triggered.

[0120] The node expansion and reduction quantity determination is based on the marginal utility analysis of the improvement amplitude of the objective function. The improvement amplitude is equal to the optimal configuration target value minus the current configuration target value. The marginal contribution of a single node is calculated by statistical data, and the marginal contribution is equal to the average improvement of the objective function when adding a node. The number of node expansions is equal to the improvement amplitude divided by the marginal contribution multiplied by the expansion safety factor 1.2, rounded up. The calculation of the number of node reductions uses a similar method, with a safety factor of 0.8 to ensure that resources are not excessively recovered. The expansion and reduction quantity calculation results need to pass the resource constraint check to ensure that they do not exceed the hardware resource upper limit. The expansion and reduction decision supports batch execution, with large-scale expansion and reduction operations divided into multiple batches to avoid shocks.

[0121] The routing weight allocation proportion conversion maps the optimal solution of the k-dimensional spectral clustering space back to the routing weight distribution of the original topology space. The conversion process uses the spatial modulation parameter for inverse transformation operation, and the inverse transformation formula is that the new weight is equal to the old weight divided by the modulation parameter power. The weight after inverse transformation is processed by row normalization to ensure that the out-degree weight sum of each node is 1. The weight allocation accuracy is reserved to 3 decimal places, and the weight update adopts a gradual method to avoid mutation, and the weight change amount of each update is not more than 20 percent of the current weight. The weight conversion module maintains the conversion history for weight change trend analysis and abnormal conversion detection.

[0122] The scaling decision scheme generation integrates all the calculation results to form a complete resource adjustment instruction. The decision scheme data structure includes five fields of decision timestamp, validity period, expanded node list, reduced node list, weight adjustment mapping and execution plan. The expanded node list records the node identifier, expansion quantity, expected start time and resource specification. The reduced node list records the node identifier, reduction quantity, expected stop time and resource recovery plan. The weight adjustment mapping is a weight dictionary from the source node to the target node, and the execution plan includes the time sequence and dependency relationship of each operation. The validity period of the decision scheme is set to 15 minutes, and it is automatically invalidated and triggers regeneration after execution timeout. The decision scheme supports rollback mechanism, which can quickly fall back to the previous state when an exception occurs during execution.

[0123] The data case demonstrates the complete implementation process. The input dynamic resource topology includes 24 processing nodes, and the node coordinates are distributed in the processing capacity range of 1 to 9 and the specialization degree range of 0.2 to 0.9. The adjacency matrix is a 24*24 sparse matrix, and the non-zero element ratio is 35%, indicating the connection density. The Laplacian matrix feature decomposition obtains five effective eigenvalues 0.12, 0.25, 0.38, 0.47, 0.59 and corresponding eigenvectors to constitute the spectral clustering space. The distribution eigenvector is a 5-dimensional vector 0.22, 0.28, 0.19, 0.16, 0.15, reflecting the distribution density of nodes in each clustering dimension. The prediction confidence 0.78 corresponds to the spatial modulation parameter 0.88, and the double target weight coefficient one is 0.65 and the weight coefficient two is 0.35. The confidence bias is negative 0.05, the spectral clustering compensation item calculation value is 0.032, and the resource redundancy threshold is 1.68. The optimization solution converges after 156 iterations, and the convergence error is 0.0008. The optimal configuration requires expanding 4 high-capacity nodes, reducing 2 low-efficiency nodes, and adjusting the routing weight of 16 connections. The generated scaling decision scheme is expected to improve the resource utilization rate from the current 68% to 80%, reduce the average processing time delay from 120 ms to 110 ms, improve the overall performance by 12%, and complete all adjustment operations in 8 minutes.

[0124] In an alternative embodiment, a risk assessment function is constructed based on the resource redundancy threshold in the scaling decision scheme, the deviation risk of resource configuration from the predicted load is quantified, and the incremental optimization of the load trend prediction model is triggered when the deviation risk breaks through the tolerance boundary, which includes: The scaling decision scheme is constructed as a robustness space, in which the coupling characteristic of the resource redundancy threshold and the actual resource configuration quantity is quantified as a resource redundancy coefficient, and an initial robustness evaluation benchmark is constructed based on the resource redundancy coefficient; In the robustness space, a risk assessment structure is constructed, the difference between the resource redundancy coefficient and the safety redundancy benchmark is mapped as the configuration robustness, the deviation of the actual event arrival rate from the predicted event arrival rate is mapped as the load robustness, and a risk assessment function is constructed through the configuration robustness and the load robustness, and a deviation risk value is generated in combination with the robustness evaluation benchmark; Based on the deviation risk value, a robustness tolerance boundary is set, the robustness tolerance boundary is dynamically adjusted using the characteristics of the robustness space, and when the deviation risk value breaks through the robustness tolerance boundary, the error characteristics of the historical load sequence are reconstructed in the robustness space; Based on the reconstructed error characteristics, a robustness optimization sample is generated, and the parameters of the load trend prediction model are incrementally optimized in combination with the evolution characteristics of the configuration robustness and the load robustness.

[0125] As shown in Figure 2 The method includes: The robustness space construction of the scaling decision scheme adopts a multi-dimensional vector space representation method, and the number of spatial dimensions is equal to the total number of resource types plus the total number of decision parameters. The resource types include four basic types of computing resources, storage resources, network resources and memory resources, and the decision parameters include four parameters of expansion quantity, reduction quantity, weight adjustment amplitude and execution time. The robustness space adopts an 8-dimensional vector representation, and each dimension corresponds to a standardized value of a resource type or a decision parameter. The space construction module receives complete information of the scaling decision scheme, and maps each resource configuration quantity and decision parameter to the standardized interval of 0 to 1. The standardization process uses the maximum and minimum value normalization method, and the maximum and minimum values are determined through statistical analysis of historical decision data.

[0126] The coupling feature quantifies the correlation and dependency strength between the resource redundancy threshold and the actual resource allocation amount. The coupling feature is measured by two indicators: Pearson correlation coefficient and mutual information. Pearson correlation coefficient reflects linear correlation, and mutual information reflects nonlinear dependency. The resource redundancy coefficient is defined as the weighted combination of the coupling feature, with weight coefficients of 0.7 and 0.3 respectively, ensuring the dominant role of linear correlation. The resource redundancy coefficient calculation module maintains a sliding window with a window length of 50 historical decision records. The resource redundancy coefficient is updated each time a new decision record is added to the window. The resource redundancy coefficient value ranges from 0 to 1, with a larger value indicating a higher degree of coupling between the resource redundancy threshold and the actual allocation amount.

[0127] The construction of the initial robustness evaluation benchmark is based on the historical distribution characteristics of the resource redundancy coefficient. The benchmark value and variation interval are determined through statistical analysis. The initial benchmark value is set to the historical median of the resource redundancy coefficient, and the variation interval is calculated by the interquartile range. The robustness evaluation benchmark adopts a dynamic updating mechanism. Whenever 10 new resource redundancy coefficient samples are accumulated, the benchmark value is recalculated. The benchmark updating adopts an exponentially weighted moving average method, with a new sample weight of 0.3 and a historical sample weight of 0.7, ensuring that the benchmark value can adapt to slow changing trends while maintaining stability. The initial robustness evaluation benchmark includes three parameters: benchmark value, upper threshold and lower threshold. The upper and lower threshold values are the benchmark value plus or minus 1.5 times the interquartile range.

[0128] The construction of the risk assessment structure in the robustness space adopts a double-layer evaluation architecture. The upper layer is the configuration robustness evaluation, and the lower layer is the load robustness evaluation. The configuration robustness evaluation module calculates the difference between the resource redundancy coefficient and the safety redundancy benchmark. The safety redundancy benchmark is determined by the mean value of the resource redundancy coefficient of the historical optimal decision. The configuration robustness is equal to the absolute value of the difference between the resource redundancy coefficient and the safety redundancy benchmark, normalized by the safety redundancy benchmark. The load robustness evaluation module calculates the relative deviation of the actual event arrival rate from the predicted event arrival rate. The deviation is calculated by the weighted combination of the root mean square error and the average absolute percentage error, with weight coefficients of 0.5.

[0129] The mapping process of the configuration robustness converts the difference between the resource redundancy coefficient and the safety redundancy benchmark into a robustness score between 0 and 1. The difference value is mapped by a piecewise linear function. When the difference is less than 10% of the benchmark value, the robustness score is 0.9 to 1.0. When the difference is between 10% and 30%, the robustness score is 0.6 to 0.9. When the difference exceeds 30%, the robustness score is 0 to 0.6. The mapping function parameters support dynamic adjustment, based on the success rate of recent configuration decisions and resource utilization efficiency. The configuration robustness calculation module records the intermediate results of each calculation, which are used for robustness trend analysis and anomaly detection.

[0130] The mapping process of load robustness converts the event arrival rate prediction error into a robustness evaluation indicator, which is calculated by a sliding window of 30 minutes containing 30 1-minute granularity event arrival rate data points. The load robustness equals to 1 minus the normalized prediction error, which equals to the current prediction error divided by the 95th percentile of historical prediction errors. The load robustness is set to 0 when the prediction error exceeds the historical 95th percentile, and is set to 1 when the prediction error is 0. The load robustness calculation adopts a multi-model ensemble method, which combines three different prediction error indicators for comprehensive evaluation. The indicator weights are determined by historical prediction performance.

[0131] The construction of the risk assessment function adopts a nonlinear combination method to weight and fuse the configuration robustness and the load robustness. The risk assessment function equals to the square of the configuration robustness multiplied by weight one plus the square of the load robustness multiplied by weight two. The default values of weight one and weight two are 0.6 and 0.4, respectively. The square operation enhances the risk assessment sensitivity when the robustness is low, and the weight setting reflects the relative importance of the configuration robustness. The risk assessment function value range is 0 to 1, and the smaller the value, the higher the risk, and the larger the value, the better the stability. The function parameters support adaptive adjustment, and the adjustment trigger condition is that the risk assessment result does not match the actual situation for 5 consecutive times.

[0132] The generation of deviation risk value compares and calculates the risk assessment function result with the robustness evaluation benchmark. The deviation risk value equals to the robustness evaluation benchmark minus the risk assessment function value, and is normalized by the robustness evaluation benchmark again. When the deviation risk value is positive, it indicates that the current risk exceeds the benchmark level, and when the deviation risk value is negative, it indicates that the current risk is lower than the benchmark level. The deviation risk value calculation precision is retained to 3 decimal places, and the calculation frequency is consistent with the load monitoring frequency, which is once per minute. The deviation risk value generation module maintains a risk history sequence, which is used for risk trend prediction and abnormal risk identification.

[0133] The setting of the robustness tolerance boundary is based on the historical distribution characteristics of the deviation risk value and the business tolerance requirement. The tolerance boundary includes two threshold values, the upper boundary and the lower boundary. The upper boundary is set to the 85th percentile of the historical distribution of the deviation risk value, and the lower boundary is set to the 15th percentile. The boundary setting considers the business continuity requirement and the resource cost constraint. During the business critical period, the tolerance boundary is narrowed, and during the business stable period, the tolerance boundary is relaxed. The tolerance boundary adopts a segmented setting strategy, which divides the 24 hours of a day into 6 periods, and sets independent tolerance boundary parameters for each period. The boundary parameters are determined by a combination of historical data statistics and expert experience.

[0134] The dynamic adjustment of the robustness tolerance boundary utilizes the statistical characteristics and variation trend information of the robustness space. The adjustment mechanism monitors the variance change and correlation evolution of each dimension in the robustness space. When the stability of the space significantly improves, the tolerance boundary is relaxed. When the volatility of the space increases, the tolerance boundary is tightened. The adjustment amplitude is controlled within 20 percent of the current boundary value, and the adjustment frequency is every 6 hours. The dynamic adjustment module uses Kalman filtering method to smooth the boundary changes, and the filtering parameters are optimized according to the historical adjustment effect. The boundary adjustment process records the adjustment reason and effect evaluation, which is used for continuous improvement of the adjustment strategy.

[0135] The detection of deviation risk value breaking the tolerance boundary adopts dual judgment criteria of continuity and severity. The continuity judgment requires that the deviation risk value exceeds the tolerance boundary for 3 consecutive times to trigger the processing flow, avoiding false triggering caused by accidental fluctuations. The severity judgment requires that the deviation risk value exceeds the tolerance boundary by more than 50 percent of the boundary value, ensuring the necessity of processing. The breakthrough detection module maintains a detection state machine, recording the breakthrough start time, duration and severity. After detection is triggered, the historical load sequence error feature reconstruction process is started immediately, and the reconstruction range is the load data of the past 7 days.

[0136] The reconstruction process of historical load sequence error features analyzes the distribution pattern and evolution law of prediction errors in the robustness space. The reconstruction module extracts the prediction error data of each hour in the past 7 days, calculates the mean, variance, skewness and kurtosis of the error. The error features in the robustness space are represented by a 4-dimensional feature vector, and each dimension corresponds to a standardized value of a statistical feature. The reconstruction process uses principal component analysis method for dimension reduction, retaining principal components with an explanation variance contribution rate of more than 95 percent. The reconstruction result contains error feature vector sequence and principal component loading matrix, which is used for subsequent optimization sample generation.

[0137] The generation of robustness optimization samples is based on the reconstructed error features and the current robustness evaluation results. The optimization sample contains input features and target labels, the input features are composed of configuration robustness, load robustness and error feature vector, and the target label is the expected prediction improvement direction. The sample generation uses the synthetic minority over-sampling technique to handle the sample imbalance problem, ensuring the balance of the number of samples of each class. The number of generated optimization samples is 100 to 500, and the specific number is determined according to the error severity and available computing resources. The sample quality is evaluated by cross-validation, and samples with a quality score lower than 0.8 are removed.

[0138] The extraction of configuration robustness and load robustness evolution features uses time series analysis methods to identify periodic and trend patterns of robustness changes. Evolutionary features include four parameters: trend slope, cycle length, amplitude size, and phase shift. The trend slope calculates the change trend of the last 30 data points through linear regression, and the cycle length identifies the dominant frequency component through Fourier transform. The amplitude size is the standard deviation of the robustness sequence, and the phase shift is determined by correlation analysis with the standard sine wave. The evolution feature extraction module maintains a feature history record for feature stability evaluation and abnormal evolution detection.

[0139] The incremental optimization of the load situation prediction model uses an online learning method based on robustness optimization samples and evolution features to update model parameters. The optimization process uses a stochastic gradient descent algorithm with a learning rate of 0.01 and a batch size of 32 samples. The model parameter update uses a momentum method with a momentum coefficient of 0.9 to prevent the optimization process from falling into a local optimum. The incremental optimization updates a subset of model weights in each iteration, with the subset size being 10% of the total number of parameters, and the updated parameters are selected through importance sampling. The optimization process monitors the validation loss changes, and when the validation loss does not decrease for 5 consecutive iterations, the optimization is stopped.

[0140] The incremental optimization process of model parameters combines the evolution features of configuration robustness and load robustness for adaptive adjustment. Evolutionary features are used to adjust the hyperparameters of the optimization algorithm, and when robustness evolution shows a stable trend, the learning rate is reduced, and when robustness evolution appears severe fluctuations, the regularization strength is increased. The optimization process uses an early stopping mechanism to prevent overfitting, and early stopping is based on the performance indicators of the independent validation set. After model updating, performance evaluation is conducted, and evaluation indicators include prediction accuracy, recall rate, and F1 score. When the performance evaluation results are lower than the historical average level, the model parameters are rolled back to the previous update.

[0141] The data case shows the complete implementation process. The input scalable decision scheme contains resource configuration information of 24 nodes, and the resource redundancy threshold is 1.68. The robustness space is constructed as an 8-dimensional vector space, and the resource redundancy coefficient is calculated as 0.75. The safety redundancy benchmark is set to 0.82, and the configuration robustness calculation result is 0.66. The average absolute percentage error between the actual event arrival rate and the predicted event arrival rate is 12.5%, and the load robustness calculation result is 0.72. The risk assessment function value is 0.68, the robustness evaluation benchmark is 0.75, and the deviation risk value is 0.093. The upper limit of the robustness tolerance boundary is set to 0.15, and the lower limit is set to -0.10. The current deviation risk value does not break through the tolerance boundary. When the simulation breaks through the scene, the deviation risk value reaches 0.18, triggering the error feature reconstruction process. The reconstructed error feature vector is a 4-dimensional vector 0.32, 0.28, 0.19, and 0.21, generating 325 robustness optimization samples. The incremental optimization process converges after 45 iterations, and the model prediction accuracy improves from 78.6% to 82.3%, with a load robustness improvement of 14.2%.

[0142] In a second aspect of the embodiments of the present application, a distributed event-driven collaborative processing system is provided, comprising: A first unit is configured to parse event stream data generated by a distributed event source, filter the event stream data into time sequence features and semantic features, fuse and encode the filtered features using a self-encoder, and construct an event multi-dimensional feature vector. A second unit is configured to construct a load situation prediction model based on the event multi-dimensional feature vector, deduce an event arrival rate curve and a prediction confidence in a future time window using the load situation prediction model, map and fuse the event arrival rate curve and the semantic features in the event multi-dimensional feature vector to generate a comprehensive load descriptor. A third unit is configured to quantify a load peak period according to the comprehensive load descriptor, group processing nodes in a resource topology according to semantic processing requirements corresponding to the load peak period, and reconstruct node numbers and routing weights according to a gradient distribution of the event arrival rate curve to generate a dynamic resource topology structure. A fourth unit is configured to map node distribution, routing weights of the dynamic resource topology structure, and the prediction confidence to an optimization space, construct a constrained problem with resource utilization rate and processing time delay as optimization objectives, evolve a resource redundancy threshold based on the prediction confidence, and generate a scalable decision scheme. A fifth unit is configured to construct a risk assessment function based on the resource redundancy threshold in the scalable decision scheme, quantify a deviation risk of resource configuration and predicted load, and trigger incremental optimization of the load situation prediction model when the deviation risk breaks through a tolerance boundary, to realize collaborative evolution of the prediction model and resource configuration.

[0143] In a third aspect, the present application provides an electronic device, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the method described above.

[0144] In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method described above.

[0145] The present application can be a method, apparatus, system, and / or computer program product. Computer program products can include computer-readable storage media having computer-readable program instructions loaded thereon for performing various aspects of the present application.

[0146] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for event-driven distributed collaborative processing, characterized in that, The method comprises the steps of: analyzing event stream data generated by a distributed event source, filtering the event stream data into time sequence features and semantic features, using an autoencoder to fuse and encode the filtered features, and constructing an event multi-dimensional feature vector; constructing a load situation prediction model based on the event multi-dimensional feature vector, using the load situation prediction model to deduce an event arrival rate curve and a prediction confidence in a future time window, mapping and fusing the event arrival rate curve and the semantic features in the event multi-dimensional feature vector to generate a comprehensive load descriptor; quantifying a load peak period according to the comprehensive load descriptor, grouping processing nodes in a resource topology according to semantic processing requirements corresponding to the load peak period, and reconstructing the number of nodes and routing weights according to the gradient distribution of the event arrival rate curve to generate a dynamic resource topology structure; mapping the node distribution, routing weights and prediction confidence of the dynamic resource topology structure to an optimization space, constructing a constrained problem with resource utilization rate and processing delay as optimization objectives, evolving a resource redundancy threshold based on the prediction confidence, and generating a scaling decision scheme; constructing a risk assessment function based on the resource redundancy threshold in the scaling decision scheme, quantifying the deviation risk of resource allocation and predicted load, triggering incremental optimization of the load situation prediction model when the deviation risk exceeds a tolerance boundary, and realizing co-evolution of the prediction model and resource allocation.

2. The method of claim 1, wherein, The method comprises the steps of: analyzing original event stream generated by a distributed event source, extracting timestamp information and event identification information of each event in the original event stream, constructing time sequence features reflecting event arrival time distribution characteristics based on the timestamp information, and constructing semantic features reflecting event types and processing requirements based on the event identification information; performing time axis segmentation processing on the time sequence features, dividing the time sequence features into a plurality of time sequence feature subsegments according to a time window, calculating the event arrival frequency and arrival interval variance in each time sequence feature subsegment to form a time sequence feature statistic set; performing semantic space mapping on the semantic features, mapping different event types to vector representations in a unified semantic space, calculating distance metrics of each event type in the unified semantic space, identifying event type clusters with similar processing requirements based on the distance metrics, and forming a semantic feature clustering result; inputting the time sequence feature statistic set and the semantic feature clustering result into a feature fusion encoder, projecting the time sequence feature statistic set by dimension reduction through the feature fusion encoder, embedding and encoding the semantic feature clustering result at the same time, tensor splicing the time sequence representation after dimension reduction and the semantic representation after embedding and encoding to obtain an event multi-dimensional feature vector.

3. The method of claim 1, wherein, constructing a load situation prediction model based on the event multi-dimensional feature vector, deducing an event arrival rate curve and a prediction confidence in a future time window by using the load situation prediction model, and fusing the event arrival rate curve and semantic features in the event multi-dimensional feature vector to generate a comprehensive load descriptor, including: separating a time sequence feature component and a semantic feature component from the event multi-dimensional feature vector, performing multi-scale time window decomposition on the time sequence feature component, and extracting a macro time sequence mode reflecting long-term evolution and a micro time sequence mode reflecting short-term fluctuation; inputting the macro time sequence mode into a trend prediction unit to obtain a baseline trend curve by trend extrapolation, inputting the micro time sequence mode into a fluctuation prediction unit to obtain a fluctuation offset by fluctuation superposition, and superimposing the baseline trend curve and the fluctuation offset to generate an event arrival rate curve; calculating a prediction stability of the trend prediction unit based on stationarity of the macro time sequence mode, calculating a prediction fluctuation degree of the fluctuation prediction unit based on randomness of the micro time sequence mode, and fusing the prediction stability and the prediction fluctuation degree to obtain a prediction confidence corresponding to the event arrival rate curve; calculating a load change rate for the event arrival rate curve, identifying a load change period in which the load change rate exceeds a preset load threshold, extracting a semantic type distribution of historical events in the period from the semantic feature component, and selecting a dominant semantic type set based on resource consumption intensity; structurally packaging the event arrival rate curve, the prediction confidence, a time identifier of the load change period, and the dominant semantic type set to generate a comprehensive load descriptor.

4. The method of claim 3, wherein, inputting the macro time sequence mode into a trend prediction unit to obtain a baseline trend curve by trend extrapolation, inputting the micro time sequence mode into a fluctuation prediction unit to obtain a fluctuation offset by fluctuation superposition, and superimposing the baseline trend curve and the fluctuation offset to generate an event arrival rate curve, including: establishing a fractal dimension mapping for the macro time sequence mode, quantifying complexity features of monotone increasing and decreasing trend segments in the macro time sequence mode, constructing a fractal predictor based on the complexity features, evolving and deducing a trend extension sequence corresponding to each trend segment in a future time window by the fractal predictor, generating a continuous baseline trend curve by optimizing the connection characteristics of trend segment boundaries through the fractal predictor; performing double feature decoupling on the micro time sequence mode, projecting the micro time sequence mode to a periodicity and randomness feature space, quantifying fluctuation period and fluctuation amplitude in the periodicity feature space, constructing a fractal predictor to reconstruct periodic fluctuation features, obtaining a periodic fluctuation sequence by iterative evolution of the reconstructed periodic fluctuation features in a future time window, establishing a fluctuation intensity description for the randomness feature space to generate a random fluctuation sequence, and adaptively combining the periodic fluctuation sequence and the random fluctuation sequence to obtain a fluctuation offset; fusing the baseline trend curve and the fluctuation offset in the time dimension to generate an event arrival rate curve containing trend evolution information and fluctuation feature information.

5. The method of claim 1, wherein, Quantifying a load peak period according to the comprehensive load descriptor, grouping processing nodes in a resource topology in combination with semantic processing requirements corresponding to the load peak period, and reconstructing the number of nodes and routing weights according to the gradient distribution of the event arrival rate curve to generate a dynamic resource topology structure includes: Establishing an entropy value mapping for the event arrival rate curve in the comprehensive load descriptor, quantifying the time interval characteristics of the event arrival rate exceeding the historical average, calculating the integral load quantity in each time interval, and determining the load peak period based on the integral load quantity; Mapping the dominant semantic type set associated with the load peak period to the entropy value space, constructing a matching entropy metric in combination with the semantic capability label of the processing node in the resource topology, reconstructing the organization structure of the processing node according to the matching entropy metric, and generating a matching node group; Constructing an entropy value mapping for the event arrival rate curve to obtain a gradient distribution curve, quantifying the characteristics of the load sudden change moment in the gradient distribution curve, deducing the resource demand increment based on the event arrival rate growth trend before the load sudden change moment, and converting the resource demand increment into the node expansion quantity of each matching node group; Reconstructing the resource distribution of each matching node group according to the node expansion quantity, quantifying the load saturation degree of the processing node in the matching node group, and constructing the routing weight based on the reciprocal of the load saturation degree; topologically encoding the organization structure, node information, quantity distribution and routing weight of the matching node group to generate a dynamic resource topology structure.

6. The method of claim 1, wherein, Mapping the node distribution and routing weight of the dynamic resource topology structure to the optimization space in combination with the prediction confidence, constructing a constrained problem with resource utilization rate and processing delay as optimization objectives, evolving the resource redundancy threshold based on the prediction confidence, and generating a scaling decision scheme includes: Constructing the node position identifier and connection relationship in the dynamic resource topology structure into a spectral clustering space, mapping the node distribution into a distribution feature vector in the spectral clustering space, constructing an association matrix based on the routing weight, and embedding the prediction confidence into the association matrix to obtain a space modulation parameter; Constructing a double-objective optimization architecture in the spectral clustering space, representing the resource utilization rate through the distribution uniformity of the distribution feature vector, representing the processing delay through the distribution characteristics of the association matrix, performing spectral clustering combination on the double-objective under the space modulation parameter to obtain a comprehensive optimization objective, and setting constraint conditions for the distribution feature vector and the association matrix in the double-objective optimization architecture; Constructing a spectral clustering compensation term based on the prediction confidence, applying the spectral clustering compensation term to the comprehensive optimization objective to generate a resource redundancy threshold, and adjusting the constraint boundary of the double-objective optimization architecture according to the resource redundancy threshold; Solving the optimal resource configuration scheme in the spectral clustering space, comparing the optimal resource configuration scheme with the current node distribution, determining the node expansion and contraction quantity based on the comprehensive optimization objective, converting the optimal resource configuration scheme into a routing weight allocation proportion according to the space modulation parameter, and generating a scaling decision scheme.

7. The method of claim 1, wherein, constructing a risk assessment function based on the resource redundancy threshold in the scaling decision scheme, quantifying the deviation risk of resource configuration and predicted load, triggering the incremental optimization of the load trend prediction model when the deviation risk breaks through the tolerance boundary, comprising: constructing the scaling decision scheme as a robustness space, quantifying the coupling feature of resource redundancy threshold and actual resource configuration quantity as a resource redundancy coefficient in the robustness space, constructing an initial robustness evaluation benchmark based on the resource redundancy coefficient; constructing a risk assessment structure in the robustness space, mapping the difference between the resource redundancy coefficient and the safety redundancy benchmark as configuration robustness, mapping the deviation between actual event arrival rate and predicted event arrival rate as load robustness, constructing a risk assessment function through the configuration robustness and the load robustness, and generating a deviation risk value in combination with the robustness evaluation benchmark; setting a robustness tolerance boundary based on the deviation risk value, dynamically adjusting the robustness tolerance boundary using the features of the robustness space, and reconstructing the error features of the historical load sequence in the robustness space when the deviation risk value breaks through the robustness tolerance boundary; generating a robustness optimization sample based on the reconstructed error features, and performing incremental optimization on the parameters of the load trend prediction model in combination with the evolution features of the configuration robustness and the load robustness.

8. An event-driven distributed collaborative processing system for implementing the method of any of the preceding claims 1-7, characterized by comprising: a first unit configured to analyze event stream data generated by a distributed event source, filter the event stream data into time sequence features and semantic features, fuse and encode the filtered features using a self-encoder, and construct an event multi-dimensional feature vector; a second unit configured to construct a load trend prediction model based on the event multi-dimensional feature vector, deduce an event arrival rate curve and a prediction confidence in a future time window using the load trend prediction model, map and fuse the event arrival rate curve and the semantic features in the event multi-dimensional feature vector to generate a comprehensive load descriptor; a third unit configured to quantify a load peak period according to the comprehensive load descriptor, group processing nodes in a resource topology in combination with semantic processing requirements corresponding to the load peak period, and reconstruct node quantity and routing weight according to the gradient distribution of the event arrival rate curve to generate a dynamic resource topology structure; a fourth unit configured to map node distribution, routing weight, and prediction confidence of the dynamic resource topology structure to an optimization space, construct a constrained problem with resource utilization rate and processing delay as optimization objectives, evolve a resource redundancy threshold based on the prediction confidence, and generate a scaling decision scheme; a fifth unit configured to construct a risk assessment function based on the resource redundancy threshold in the scaling decision scheme, quantifying the deviation risk of resource configuration and predicted load, triggering the incremental optimization of the load trend prediction model when the deviation risk breaks through the tolerance boundary, and realizing the co-evolution of the prediction model and resource configuration.

9. An electronic device, comprising: comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to execute the method of any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon computer program instructions, wherein, The computer program instructions, when executed by a processor, implement the method of any one of claims 1 to 7.

Images