A heterogeneous computing power cluster intensive security management method and system

Abstract

The application discloses a heterogeneous computing power cluster intensive security management and control method and system, belongs to the field of computing power resource scheduling and network security, and relates to the technical field of computer network security. The method comprises the following steps: acquiring a calling request for a first heterogeneous computing power cluster; querying a preset mapping data structure based on a first credential to obtain a second credential corresponding to the first credential and a characteristic portrait bound with the first credential; performing cross-heterogeneous environment cost normalization on request load data; when the pre-checking is passed, establishing an isolated channel with the first heterogeneous computing power cluster by using the second credential to forward the calling request; when the cumulative consumption exceeds the estimated quota level, the forwarding is stopped and the isolated channel is released, otherwise, a standardized audit log is output after the transmission is completed.

Description

Technical Field

[0001] This invention relates to the fields of computing power resource scheduling and network security, and in particular to a method and system for centralized security management of heterogeneous computing power clusters. Background Technology

[0002] With the rapid evolution of large-scale AI model technology, the way various tenant business systems consume underlying computing resources has evolved from single-cluster invocation to collaborative invocation of heterogeneous computing clusters across vendors, hardware, and billing standards. In typical heterogeneous computing resource invocation scenarios, the same platform needs to simultaneously connect to multiple types of computing resources, such as private graphics processing unit clusters, external commercial large-scale model application programming interface services, and local private inference clusters. These various computing resources differ significantly in terms of authentication systems, billing units, and response characteristics. Existing heterogeneous computing power invocation solutions typically expose the authentication information of downstream computing power clusters directly to the tenant side at the gateway layer, or through shallow encapsulation. Furthermore, the billing methods for each downstream cluster are fragmented and settled independently. In practical applications, this approach presents several problems: 1. The physical authentication credentials of downstream computing power clusters are at risk of being reverse-engineered. There is a lack of mandatory decoupling between the abstract identity perceived by the tenant and the physical authentication credentials perceived only by the downstream cluster. The tenant may reverse-engineer the physical authentication credentials through protocol layer interception or parameter layer callbacks, thereby bypassing the gateway and directly invoking the downstream cluster, compromising the authentication security of the platform's computing power assets. 2. Cost metrics across heterogeneous clusters cannot be pre-controlled within a unified framework. The input and output billing prices of different computing power clusters differ by orders of magnitude, and there is a lack of unified dimensionless metrics. This prevents the gateway layer from pre-checking and intercepting the expected consumption of a request before it enters the downstream cluster. Once the request enters the downstream cluster, it is completely exposed to the risk of excessive consumption. Summary of the Invention

[0003] One of the objectives of this invention is to provide a centralized security management method for heterogeneous computing power clusters, in order to solve the problems in the prior art where physical authentication credentials are easily obtained by reverse engineering, cost calibers across heterogeneous clusters cannot be pre-controlled under a unified framework, and existing audit log systems cannot pre-identify abnormal behavior.

[0004] This invention is achieved through the following technical solution: a method for centralized security management of heterogeneous computing power clusters, comprising the following steps: obtaining a call request for a first heterogeneous computing power cluster, the call request including a first credential and request load data, wherein the first credential does not carry authentication information of the first heterogeneous computing power cluster; querying a preset mapping data structure based on the first credential to obtain a second credential corresponding to the first credential and a feature profile bound to the first credential, the second credential being used to complete authentication with the first heterogeneous computing power cluster, the feature profile including at least dynamic features reflecting the real-time consumption rate of the first credential; and processing the request load data... Cost normalization across heterogeneous environments is used to determine the estimated quota level of the call request, and pre-verification is performed based on the estimated quota level and the feature profile. When the pre-verification passes, an isolation channel is established with the first heterogeneous computing power cluster using the second credential to forward the call request. During the process of the first heterogeneous computing power cluster returning streaming response data through the isolation channel, the cumulative consumption of the call request, which is of the same dimension as the estimated quota level, is accumulated in real time. When the cumulative consumption exceeds the estimated quota level, forwarding is stopped and the isolation channel is released; otherwise, a standardized audit log is output after the transmission is completed.

[0005] Furthermore, the request payload data includes at least one of the following: a token sequence or byte stream of the input context, model selection parameters, generation control parameters, a switch flag indicating whether to enable streaming response, and tenant-defined metadata fields.

[0006] Furthermore, the first credential includes any one of the following: a signature string conforming to the JWT specification, an irreversible mapping key value generated based on a UUID, or a composite identity handle bound to the tenant's business system.

[0007] Furthermore, the centralized security management method further includes: extracting historical time consumption sequences from the standardized audit log using a composite index of the first credential and the first heterogeneous computing power cluster; maintaining a dynamic baseline mean and a dynamic baseline deviation for the composite index based on exponential moving average calculation, wherein the dynamic baseline mean is obtained by weighting the latest time consumption observation with the previous mean using a smoothing coefficient, and the dynamic baseline deviation is obtained by weighting the squared residual of the latest time consumption observation with the previous mean using the smoothing coefficient and the previous deviation; using the sum of integer multiples of the dynamic baseline mean and the dynamic baseline deviation as a dynamic anomaly threshold; and determining that an anomaly alarm is triggered when a time consumption observation in the historical time consumption sequence exceeds the dynamic anomaly threshold and the real-time consumption rate in the feature profile exceeds a preset high-frequency judgment threshold.

[0008] Furthermore, the dynamic baseline mean and the dynamic baseline deviation are maintained independently at the granularity of the composite index of the first certificate and the first heterogeneous computing power cluster, and the dynamic baseline mean and the dynamic baseline deviation under different composite indices are isolated from each other.

[0009] Furthermore, the conditions for determining to trigger an abnormal alarm are: the time consumption observation values ​​of a consecutive preset number of times in the historical time consumption sequence all exceed the dynamic abnormal threshold, and the real-time consumption rate exceeds the high-frequency determination threshold.

[0010] Furthermore, after determining that an abnormal alarm has been triggered, the method further includes: removing or marking the composite data object corresponding to the first credential in the mapping data structure as invalid from the mapping data structure, so that subsequent call requests initiated by the first credential are intercepted in the step of querying the preset mapping data structure based on the first credential.

[0011] Further, obtaining the call request for the first heterogeneous computing power cluster includes: completing the transport layer security handshake and protocol decapsulation of the call request through a reverse proxy component; performing layered stripping of the request header field and request body on the decapsulated call request to extract the first credential from the request header field and the request payload data from the request body; and writing the first credential and the request payload data into different data areas in the memory of the processing device.

[0012] Furthermore, the mapping data structure is a key-value data structure that resides in the high-speed memory of the processing device. Its key is the hash digest of the first credential, and its value is a composite data object that includes the second credential, permission level, global total quota limit, and historical behavior feature fingerprint.

[0013] Furthermore, the mapping data structure is hosted within a Redis instance.

[0014] Furthermore, the mapping data structure is carried in a hash table within the local process.

[0015] Furthermore, the second credential includes any one of the following: the native API Key of the external business model service, the OAuth access token of the private computing power scheduling platform, or the client certificate fingerprint based on bidirectional transport layer security.

[0016] Further, based on the first credential query, a preset mapping data structure is used to obtain a second credential corresponding to the first credential and a feature profile bound to the first credential, including: performing an irreversible hash operation on the first credential to obtain a mapping query key; performing a constant-time lookup operation in the mapping data structure using the mapping query key to obtain the composite data object; and extracting the second credential from the composite data object separately and temporarily storing it in a protected memory area.

[0017] Furthermore, the protected memory region is established by a system call that marks individual memory segments as read-only.

[0018] Furthermore, the protected memory region is carried by a cryptographic enclave built through a hardware trusted execution environment.

[0019] Furthermore, the second credential is written into a read-only secure register that is invisible to external requesters during its lifetime. During the process of establishing an isolated channel with the first heterogeneous computing power cluster using the second credential, the system's underlying network socket directly extracts the second credential from the read-only secure register through a privileged system call and encapsulates the request header field. The privileged system call completes the encapsulation in kernel mode and does not generate an addressable copy of the second credential in user-mode memory.

[0020] Furthermore, the feature profile also includes: static features reflecting the inherent attributes of the first credential, the static features including at least one of permission level, global total quota limit, and historical risk fingerprint score; the dynamic features also include the remaining resource quota level.

[0021] Furthermore, the real-time consumption rate is obtained by weighting and summing the resource consumption of each request event triggered by the first credential within a preset sliding time window by applying a time decay factor, and then normalizing it with the width of the sliding time window; the time decay factor decreases exponentially as the time difference between the request event and the current time increases, so that the closer the request event is to the current time, the higher its contribution to the real-time consumption rate.

[0022] Furthermore, resource consumption is measured in tokens or bytes.

[0023] Further, cost normalization across heterogeneous environments is performed on the request load data to determine the estimated quota level of the call request, including: extracting the input length from the request load data, and obtaining an adaptive output length estimate based on the historical input-output length statistics corresponding to the first credential, wherein the adaptive output length estimate does not exceed the maximum output length limit corresponding to the first credential; obtaining the expected original cost of this request based on the input length, the adaptive output length estimate, and the input-output billing unit price matrix of the first heterogeneous computing power cluster; and converting the expected original cost into the dimensionless estimated quota level using the platform's global benchmark exchange rate constant.

[0024] Further, the pre-verification based on the estimated quota level and the feature profile includes: determining whether the remaining resource quota level in the feature profile is sufficient to cover the estimated quota level, and obtaining a first Boolean judgment result; determining whether the real-time consumption rate in the feature profile is lower than a preset safety rate threshold, and obtaining a second Boolean judgment result; only when both the first Boolean judgment result and the second Boolean judgment result are true, the pre-verification is determined to pass, otherwise the pre-verification is determined to fail and the call request is intercepted.

[0025] Furthermore, the estimated quota level is also superimposed with a load premium term that is positively correlated with the real-time load congestion rate of the first heterogeneous computing power cluster, so that the higher the real-time load congestion rate of the first heterogeneous computing power cluster, the larger the estimated quota level.

[0026] Furthermore, the real-time load congestion rate is obtained by aggregating the processor utilization and queue backlog of the first heterogeneous computing power cluster periodically collected.

[0027] Furthermore, the security rate threshold is dynamically configured based on the permission level in the feature profile; the higher the permission level, the more lenient the corresponding security rate threshold.

[0028] Further, before establishing an isolated channel with the first heterogeneous computing power cluster using the second credential, the method further includes: periodically acquiring the real-time response latency and active concurrent connection count of each candidate computing power cluster in the candidate computing power cluster set, and constructing a real-time state vector for each candidate computing power cluster; determining a step penalty factor for each candidate computing power cluster, wherein the step penalty factor is 1 when the real-time response latency of the candidate computing power cluster does not exceed a preset response latency threshold and is 0 when it exceeds the response latency threshold; obtaining the routing weight of each candidate computing power cluster by using the permission level in the feature profile as the numerator, the comprehensive resistance value obtained by applying a weighted nonlinear norm to the real-time state vector as the denominator, and the step penalty factor as the pre-multiplier; and selecting the cluster with the largest routing weight from the candidate computing power cluster set as the first heterogeneous computing power cluster.

[0029] Furthermore, the isolation channel reuses the transmission control protocol connection pool between the processing device and the first heterogeneous computing power cluster at the physical layer, and performs tenant isolation at the logical layer according to the first credential and request context.

[0030] Furthermore, the weighted nonlinear norm is a p-norm operation performed after applying adjusted weights to each dimension of the real-time state vector, where p takes a value greater than 1.

[0031] Furthermore, the denominator is further superimposed with a positive bias term to prevent numerical singularities.

[0032] Furthermore, the real-time accumulation of the cumulative consumption of the call requests, which is of the same dimension as the estimated quota level, includes: initializing an interception buffer pool and a step accumulator with an initial value of zero for the call requests; monitoring each data frame of the streaming response data, segmenting the data frame using a preset deterministic segmentation algorithm to obtain the effective computing unit number of the data frame; using the output billing unit price of the first heterogeneous computing power cluster and the platform's global benchmark exchange rate constant, converting the effective computing unit number into a single-frame consumption of the same dimension as the estimated quota level, and accumulating it to the step accumulator, using the current value of the step accumulator as the cumulative consumption.

[0033] Furthermore, when the accumulated consumption exceeds the estimated quota level, forwarding is stopped and the isolated channel is released, including: using a hardware-level comparator to compare the accumulated consumption with the estimated quota level, driving the state machine to make a deterministic transition between the forwarding state and the circuit breaker discarding state; when the accumulated consumption first exceeds the estimated quota level, the state machine jumps to the circuit breaker discarding state, discards the remaining unforwarded data frames in the interception buffer pool, actively terminates the isolated channel by sending a connection reset message, and returns an abnormal status code of quota exhaustion to the tenant side; when the accumulated consumption does not exceed the estimated quota level at the end of the streaming response data transmission, the state machine remains in the forwarding state and enters the output of the standardized audit log.

[0034] Furthermore, the number of effective computing units is the number of output tokens or output bytes within the data frame.

[0035] Furthermore, the output of standardized audit logs includes: capturing non-standard operating status messages returned by the first heterogeneous computing power cluster in real time via a probe module deployed via a bypass; extracting audit metadata from the non-standard operating status messages according to a preset protocol parsing rule tree, wherein the audit metadata includes at least one of the following: a unique request identifier, millisecond-level consumption time, actual number of settlement computing units, and original status code; converting the original status code into a globally unified standard response status code using a preset mapping lookup table; and combining the converted audit metadata into the standardized audit log according to a fixed schema and persistently storing it.

[0036] Furthermore, the centralized security management method also includes: when the number of consecutive connection establishment timeouts for each candidate computing power cluster in the candidate computing power cluster set exceeds a preset connection establishment timeout tolerance, marking the call request as entering degradation, canceling ongoing connection retry, and releasing the memory buffer and file descriptor already occupied by the call request; retrieving the pre-loaded local degradation configuration from the local non-volatile memory of the processing device, and sending a static degradation response packet containing cluster circuit breaker prompt information to the tenant side according to the local degradation configuration.

[0037] Furthermore, the static degradation response packet is a service unavailability status response message carrying a preset JSON response body.

[0038] Another aspect of the present invention provides a heterogeneous computing power cluster centralized security management system, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements any of the heterogeneous computing power cluster centralized security management methods described above.

[0039] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0040] 1. This invention establishes a closed-loop, integrated security management framework at the gateway layer, which includes mandatory decoupling of abstract credentials and physical authentication credentials, cost normalization measurement across heterogeneous environments, and real-time economic control during the streaming process. This framework enables a mandatory decoupling mapping between the first credential perceptible to the tenant and the second credential perceptible only to the downstream cluster at the gateway layer. This ensures that no matter how the terminal tenant interacts with the gateway, it cannot obtain the physical authentication credentials through the protocol or parameter layers, thus guaranteeing the authentication security of the platform's computing power assets from a physical perspective.

[0041] 2. This invention unifies the cost of different heterogeneous clusters, expressed in their respective physical dimensions, into a dimensionless estimated quota level, thereby enabling the comparison, accumulation, and controllability of costs across heterogeneous clusters. This lays the foundation for pre-verification of requests before they enter downstream clusters within a unified sandbox. Furthermore, by comparing the accumulated consumption with the estimated quota level in real time during streaming response transmission, a deterministic jump is driven when the quota level is exceeded to forcibly stop forwarding and release the isolation channel. This deterministic approach eliminates excessive economic losses in streaming scenarios and overcomes the shortcomings of existing solutions that can only deduct fees post-processing.

[0042] 3. This invention introduces a time decay factor that decreases exponentially over time into the statistics of real-time consumption rate, so that the closer the request event is to the current moment, the higher its contribution to the rate statistics. This enables the invention to sensitively capture sudden surges in calls within a short period of time and overcomes the averaging defects of the traditional fixed time window counting method. By introducing a dynamic routing weight mechanism with a veto step penalty factor, the weight of a downstream node is forcibly reduced to zero when the response delay of the downstream node exceeds the tolerance threshold, and it is completely removed from the routing candidate set, thus avoiding the formation of a request black hole. Attached Figure Description

[0043] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0044] Figure 1 This is a flowchart of the method provided in Embodiment 1 of the present invention.

[0045] Figure 2 The graph shows the effect of different time decay constants on the sensitivity of the rate curve and the forgetting rate, as provided in Embodiment 1 of the present invention.

[0046] Figure 3 This is a schematic diagram of adaptive output length estimation provided in Embodiment 1 of the present invention.

[0047] Figure 4This is a bar chart comparing the original cost and the unified quota level of the same manufacturer, as provided in Embodiment 1 of the present invention.

[0048] Figure 5 The economic inhibition curve provided in Embodiment 1 of the present invention.

[0049] Figure 6 This is a two-dimensional decision boundary heatmap of the sandbox interception determination function provided in Embodiment 1 of the present invention.

[0050] Figure 7 The graph shows a comparison between the step penalty function provided in Embodiment 1 of the present invention and the traditional linear decay function.

[0051] Figure 8 This is a comparison chart of traffic transfer speeds for different routing strategies provided in Embodiment 1 of the present invention.

[0052] Figure 9 This is a comparison chart of excess economic loss exposure provided in Embodiment 1 of the present invention.

[0053] Figure 10 This is a comparison chart of the time-consuming adaptive capabilities of cross-heterogeneous models provided in Embodiment 1 of the present invention.

[0054] Figure 11 This is a diagram illustrating the false alarm suppression effect of the Boolean decision function provided in Embodiment 1 of the present invention.

[0055] Figure 12 This is a diagram illustrating the protection effect against extreme network partitions provided in Embodiment 1 of the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0057] Example 1

[0058] This embodiment discloses a method for intensive security management of heterogeneous computing power clusters. Figure 1 The overall method flowchart of this embodiment is shown. As can be seen from the figure, this embodiment includes the following steps:

[0059] Step 1: Receive heterogeneous computing power call requests initiated from the tenant side, parse the call requests, and extract the virtual identity token and request payload data containing the request context.

[0060] Heterogeneous computing power invocation requests refer to service requests initiated proactively by tenant-side applications, terminal software, or third-party business systems through standard HTTP / HTTPS protocols or streaming SSE protocols to the front-end security gateway, intending to consume downstream heterogeneous computing power resources. For example, a heterogeneous computing power invocation request could be a dialogue completion request for a large language model, a multimodal request for an image generation model, an inference request for a local private GPU cluster, or a request to invoke an external commercial API.

[0061] A virtual identity token is an abstract credential pre-assigned by the platform to tenants, used to uniquely identify tenants at the gateway layer and trigger permission mapping, and completely decoupled from the authentication key of the downstream real physical computing power cluster. For example, a virtual identity token can be a signature string conforming to the JWT specification, a random irreversible mapping key value generated based on UUID, or a composite identity handle deeply bound to the tenant's business system. Its core feature is that the credential itself does not carry any real authentication information from underlying heterogeneous computing power vendors (such as private GPU clusters or external business model APIs), thus semantically completely isolated from physical layer authentication.

[0062] Request payload data, which includes the request context, refers to the set of all business parameters in the request body, excluding identity credentials, used to fully reconstruct the input semantics of this call. Request payload data includes, but is not limited to, the token sequence or byte stream of the input context, model selection parameters, generation control parameters (such as temperature coefficient, Top-K, maximum output length limit), a switch flag indicating whether streaming response is enabled, and tenant-defined metadata fields.

[0063] Request context refers to the key statistical information further extracted from the request payload, used to drive subsequent cost normalization and security decisions. Request context includes, but is not limited to, the length of the input data. Request initiation time The tenant session identifier and the target model label declared in this request.

[0064] In this embodiment, heterogeneous computing power call requests initiated by the tenant can be automatically encapsulated and sent by the software development kit (SDK) deployed on the tenant's environment side, or the tenant can construct and send a standard HTTP request itself.

[0065] In this embodiment, when the gateway receives a call request, it first completes the TLS handshake and protocol decapsulation through the reverse proxy component, and then hands it over to the request parser to perform layered stripping of the request method, request path, request header fields and request body. The virtual identity token is extracted from the 'Authorization' request header field, and the request payload data is extracted from the request body. The extracted two are written to different data areas in the gateway's memory to support subsequent decoupling mapping and cost normalization operations.

[0066] Step 2: Query the preset isolation mapping dictionary in the gateway memory, decouple the virtual identity token and map it to the real physical authentication key for the target heterogeneous computing power cluster, and extract the token profile bound to the virtual identity token.

[0067] The isolation mapping dictionary is a key-value data structure residing in the gateway's high-speed memory (e.g., a Redis instance or a local in-process memory hash table). Its key is a hash digest of the virtual identity token, and its value is a composite data object containing the real physical authentication key, tenant permission level, global total quota limit, and historical behavioral fingerprints. This dictionary is designed to forcibly decouple the tenant-perceptible abstract identity from the physical authentication credentials perceptible only to the downstream heterogeneous computing cluster at the gateway layer. This ensures that regardless of how the end tenant interacts with the gateway, it cannot obtain the real physical authentication key through the protocol or parameter layers.

[0068] A true physical authentication key refers to an original credential issued by a downstream heterogeneous computing cluster vendor (e.g., a private GPU cluster management system, an external commercial large-scale model API service provider, etc.) and used to complete legitimacy verification within its own identity authentication system. For example, a true physical authentication key can be the native APIKey of an external commercial model API, an OAuth access token of a private GPU scheduling platform, or a client certificate fingerprint based on mTLS.

[0069] A token profile is a comprehensive feature vector that aggregates all current security decision bases for a virtual identity token as the primary key. The token profile includes two main categories of features: static and dynamic dimensions. Static dimensions include tenant permission level, global total quota limit, historical risk fingerprint score, etc., which reflect the inherent attributes and service level of the tenant within the platform. Dynamic dimensions include real-time consumption rate calculated based on a sliding time window, remaining resource quota level, etc., which reflect the intensity of the tenant's real-time behavior and the amount of available resources at the current moment.

[0070] In this embodiment, the virtual identity token is decoupled and mapped to a real physical authentication key for the target heterogeneous computing power cluster, which may specifically include:

[0071] Perform an irreversible hash operation on the virtual identity token to obtain the mapping query key;

[0072] Perform a constant-time lookup operation in the isolation mapping dictionary using the mapping lookup key to obtain a composite data object containing the real physical authentication key, tenant permission level, global total quota limit, and historical behavior feature fingerprint;

[0073] The actual physical authentication key is extracted separately and temporarily stored in the protected area of ​​the gateway's memory, for use only in subsequent step 4 when establishing a logical isolation channel.

[0074] Understandably, to prevent the real physical authentication key from being reverse-engineered by external requesters through memory scanning, parameter callbacks, or debugging interfaces while it is in the gateway's memory, the mapping operation is only performed in the gateway's protected memory area. The real physical authentication key is written to a read-only secure register that is invisible to external requesters throughout its lifecycle. When concatenating the real physical request header sent to the target heterogeneous computing power cluster, the underlying network socket of the system directly extracts the real physical authentication key from the read-only secure register and encapsulates it to ensure that end users cannot obtain the key through packet capture or parameter callbacks.

[0075] Traditional fixed-time-window counting methods treat all requests within a time window as having equal weight and reset them to zero at the end of the window. This mechanism suffers from a serious averaging defect. It mixes short-term surges in crawling with evenly distributed normal access, resulting in similar count values ​​and failing to effectively distinguish between sudden DDoS attacks or malicious low-frequency probing behaviors.

[0076] To overcome the aforementioned shortcomings, this embodiment introduces a rate statistics method that is highly sensitive to time and location. The core idea is to assign different weights to each historical request record based on its distance from the current time when calculating the total number of requests within the sliding window. More recent requests should be given higher weights, while the impact of older requests should decay exponentially over time. Similar to the exponential decay process, by introducing a time decay factor, recent sudden events are amplified in the calculation results, while the interference of normal historical accesses from the distant past on the current results is naturally suppressed.

[0077] Specifically, for a virtual identity token In the sliding window Each request event that occurs within the time frame can be determined based on the time elapsed since the request. The time difference is used to apply an exponentially decaying weight to the consumption, and then the weighted consumption of all events within the window is summed, with the summation based on the window width. After normalization, a real-time rate estimate that truly reflects the intensity of behavior in the vicinity at the current moment is obtained.

[0078] For example, in this embodiment, time For virtual identity tokens Real-time consumption rate It can be calculated using the following formula:

[0079]

[0080] in, In the sliding window Internal token The collection of all request events that are triggered; For a single event The resource consumption can be the number of tokens or the number of bytes; For the event The moment it happened; The time decay constant ( Its physical meaning is to control the rate at which historical behavior is forgotten; The width of the sliding time window is used to normalize the cumulative weighted consumption, so that... A physical quantity having the dimension of consumption per unit time. Figure 2 The diagram illustrates the effect of different time decay constants on the sensitivity of the rate curve and the forgetting rate in this embodiment. It can be seen that the larger the time decay constant, the steeper the rise and the sharper the peak during the sudden change, but the faster the decay and fall back. The smaller the time decay constant, the smoother the curve, and the longer the residual effect of historical behavior lasts.

[0081] Understandably, when Approaching the current moment At that time, the attenuation factor When the value approaches 1, the corresponding request participates in the statistics with almost full weight; when The further back in time the factor is from the present moment, the more exponentially it approaches zero, and its historical influence is rapidly erased. This allows the formula to capture sudden surges in requests within a short period of time with extreme sensitivity, fundamentally overcoming the inherent flaw of traditional sliding windows in averaging out spikes.

[0082] In addition, based solely on real-time rate This single dimension is insufficient to make a comprehensive security assessment of a token. Different tokens may differ significantly in terms of tenant permission levels, historical risk records, and remaining resource quotas; these static attributes are also important bases for security decisions. Therefore, it is necessary to further construct a comprehensive profile feature vector that integrates features from multiple dimensions, building upon the real-time data rate, so that it can be uniformly called and referenced in subsequent sub-models.

[0083] For example, in this embodiment, the updated comprehensive portrait feature vector It can be represented as follows:

[0084]

[0085] in, For tokens The permission level of the corresponding tenant reflects its service level within the platform and is used as a permission multiplier in subsequent routing weight and security threshold configurations. The real-time decay rate calculated by the above formula is the only dynamic component in the image vector, which is updated in real time with each new request. The risk fingerprint score is calculated based on the historical behavior of the token and is used to quantify the token's criminal record. Specifically, it can be obtained by weighted aggregation of indicators such as the number of alarms triggered in the past, the number of circuit breakers in the past, and the abnormal request rate of the token. The current remaining resource quota level of the token is the core constraint for determining whether to allow the request. This is the transpose symbol for a matrix.

[0086] Understandably, this four-dimensional feature vector organically integrates dynamic perception with static attributes, providing a unified token state input interface for the entire centralized security management system. The generated comprehensive profile feature vector is then structured and written into the gateway cache, with the key being the hash digest of the virtual identity token and the value being the serialized byte string of the vector, so that it can be read in constant time complexity in subsequent steps of the same request or in subsequent requests from the same tenant.

[0087] Step 3: Perform cost normalization calculation across heterogeneous environments on the request load data, generate the estimated security quota level for the current request, and perform pre-quota sandbox verification based on the estimated security quota level and the real-time consumption rate threshold in the token profile.

[0088] Cost normalization calculation across heterogeneous environments refers to the process of uniformly converting the expected physical cost of a request into a dimensionless security quota level defined within the platform through a series of parameterized mapping operations, before the request is actually forwarded to the downstream heterogeneous cluster. A prominent feature of heterogeneous computing power clusters is the extreme unevenness of unit computing power value. The cost of calling a local private GPU cluster differs by orders of magnitude from the cost of calling an external commercial API (such as GPT-4); the input billing price and output billing price of different clusters are also completely different. If a unified normalization process is not performed at the gateway layer, the billing standards of each heterogeneous node will be fragmented, making it impossible to perform cross-node quota verification and security interception within a unified security sandbox framework.

[0089] The estimated safe quota level refers to a dimensionless numerical indicator, obtained after cost normalization calculation, that represents the expected deduction from the platform's internal quota account for this request. This indicator is both a core input for the pre-screening sandbox verification and a hard upper limit for the streaming economic circuit breaker in subsequent step 5.

[0090] Pre-quota sandbox verification refers to the process by which the gateway performs two rounds of Boolean judgment on a request before it enters the downstream cluster, based on the estimated security quota level and the real-time consumption rate threshold in the token profile. Only if both rounds of judgment pass simultaneously can the request be allowed to proceed to the next stage.

[0091] In this embodiment, in order to accurately estimate the actual resource consumption of a request before it is actually forwarded to the downstream heterogeneous cluster, and to perform a legality determination on a uniform scale, thereby eliminating the differences in billing units between different vendors and different hardware, the specific logic of cost normalization calculation across heterogeneous environments is divided into the following three steps:

[0092] The first step is to address the issue of output length estimation. Since the output length of the AI ​​model is unknown at the time of request initiation, and the cost of the output token is often higher than that of the input token, a reasonable estimate must be made before the request is initiated. The estimation approach is as follows: based on the statistical patterns between the input and output lengths in the tenant's historical requests, a linear regression is used to fit an expected output coefficient for the specific tenant. Then, the estimated result is constrained by the maximum allowed output length of the token to prevent meaningless overflow of the estimated value.

[0093] For example, in this embodiment, the adaptive output length estimate is... It can be calculated using the following formula:

[0094]

[0095] in, The length of the input context for this request is obtained from the request payload data in step 1; For tokens The maximum allowed output length is determined by the permission level of the tenant corresponding to the token. Decide; For tenant-based The slope coefficient of the input-output length linear regression obtained by fitting historical request data represents the tenant's personalized tendency in output generation behavior; The corresponding intercept term reflects the expected basic output length of the tenant under extremely short input conditions. It is understandable that comparing the linear fitting result with the hard upper bound using the min operator effectively prevents meaningless overflow of the estimated output length in extreme cases of abnormally long inputs (such as maliciously long context injection), thus ensuring the numerical stability of subsequent cost normalization calculations. Figure 3 A schematic diagram of adaptive output length estimation in this embodiment is shown.

[0096] The second step addresses the cost normalization problem across heterogeneous clusters; after obtaining the input length and estimated output length, it can be combined with the target cluster. Using a unique input-output unit price matrix, the estimated original cost of this request is calculated. Then, a platform-wide benchmark exchange rate constant is introduced. This physical cost is converted into a uniform, dimensionless safety quota level within the system, thereby eliminating the differences in billing units between different manufacturers and different hardware.

[0097] At the same time, it is necessary to consider the potential load congestion of heterogeneous clusters at different times. When a cluster is congested, its effective service capacity decreases, and continuing to forward a large number of requests to it will exacerbate resource waste and latency. Therefore, a load premium penalty proportional to the real-time congestion rate should be added to the normalized cost to artificially increase the equivalent cost of the cluster in quota level estimation, thereby generating a natural economic inhibition effect in subsequent scheduling decisions.

[0098] For example, in this embodiment, for heterogeneous clusters Standardized estimated safety quota level It can be calculated using the following formula:

[0099]

[0100] in, and Heterogeneous clusters The dedicated input and output billing unit prices constitute the billing matrix of this cluster, and their values ​​can be read from the real-time billing rate table maintained within the gateway. As the platform's global benchmark exchange rate constant, its function is to uniformly map the costs of various heterogeneous clusters, expressed in physical world units (such as GPU card time, APIToken billing), to the security quota unit defined within the system, so that the costs of different clusters are comparable. For cluster exist Real-time load congestion rate at any given time (range of values) This can be obtained by aggregating indicators such as CPU / GPU utilization and queue backlog collected periodically by cluster health probes; This is the load premium penalty coefficient, used to control the sensitivity of congestion to the amplification of equivalent costs. Figure 4 The bar chart showing the comparison between the original costs of different vendors and the unified quota level for cost normalization across heterogeneous clusters in this embodiment is shown. It can be seen that the platform's global benchmark exchange rate constant can smooth out the billing standards with orders of magnitude differences between different vendors and different hardware, and realize cross-node quota verification under a unified sandbox framework. Figure 5 The diagram shows the economic suppression curve of the load premium penalty coefficient on the equivalent cost in this embodiment.

[0101] Understandably, when When the premium item degenerates to zero, the normalized cost is determined solely by the basic billing; when At that time, the equivalent cost was magnified to the original. The cluster was significantly suppressed in subsequent resource allocation competition, thus achieving the scheduling effect of naturally guiding traffic to avoid congested nodes through economic means.

[0102] The third step involves making a preliminary interception decision based on the normalized estimated quota level; after obtaining... Then, pre-interception can be performed before the request enters the downstream cluster. The logic for this judgment is: only if the remaining token quota... A request is only allowed to proceed to the next stage through the sandbox if the estimated normalized cost of the current request is sufficient to cover the current token's consumption rate and the current token's real-time consumption rate does not exceed the safe rate threshold.

[0103] For example, in this embodiment, the sandbox interception determination function It can be represented as follows:

[0104]

[0105] in, This is an indicator function that takes the value 1 (allows passage) when the logical condition within the parentheses is true, and takes the value 0 (blocks passage) otherwise. The remaining quota level in the token profile of step 2; The real-time decay rate in the token profile of step 2; This is the maximum allowed security rate threshold, which can be adjusted based on the different tenant's permission level. Dynamic configuration: higher-privilege tenants are assigned a more lenient rate threshold, while ordinary tenants are assigned a more stringent rate threshold. Figure 6 The two-dimensional decision boundary heatmap of the sandbox interception decision function in this embodiment is shown.

[0106] It should be noted that these two Boolean conditions are expressed by a logical AND (... The connection means that both sufficient quota and rate compliance must be met simultaneously for a request to be allowed; failure to meet either condition will trigger a pre-blocking mechanism. This logic is designed so that: sufficient quota alone cannot prevent token hijacking and high-frequency attacks, and rate compliance alone cannot prevent a single large-parameter request from exhausting the account. Only when both conditions are met simultaneously can the request be ensured to have both economic and behavioral legitimacy.

[0107] Step 4: When the current quota sandbox verification is passed, calculate the dynamic routing weight based on the token profile and the current running status of the target heterogeneous computing power cluster, and use the real physical authentication key to establish a logical isolation channel to forward the call request.

[0108] Dynamic routing weight refers to a scalar indicator that uses the permission level in the token profile as the numerator and the real-time state resistance function of the target cluster as the denominator. It serves as a quantitative basis for comprehensive selection among multiple candidate heterogeneous computing power clusters.

[0109] A logically isolated channel refers to a data transmission channel established by a gateway with a target heterogeneous computing power cluster using a real physical authentication key. Physically, it uses the same underlying TCP connection pool, but logically, it strictly isolates tenants according to virtual identity tokens.

[0110] In this embodiment, to overcome the shortcomings of traditional static load balancing strategies (e.g., round-robin or fixed weights) which lack real-time node status awareness and continue to forward requests to heterogeneous nodes even when they enter a degraded state due to response delays or concurrency saturation, thus creating a request black hole, the step of calculating dynamic routing weights specifically includes:

[0111] After the pre-sandbox determines and confirms that the request is legitimate and has sufficient quota, the next core challenge is how to dynamically and intelligently select the optimal target cluster from multiple heterogeneous backend nodes with varying states. Traditional static load balancing strategies lack the ability to perceive the real-time state of nodes. When a heterogeneous node enters a degraded state due to delayed response or concurrency saturation, static routing will continue to forward requests to it, creating a request black hole and causing a sharp deterioration in system performance.

[0112] The modeling approach to solve this problem needs to consider two levels of issues: First, the denominator of the routing weight should be able to sense the real-time health status of the node, and the routing weight should decrease more drastically as the node load increases and the response slows down through a non-linear function. Second, for nodes that have exceeded the response latency tolerance threshold, it is not enough to simply reduce their weight; a veto should be implemented to force their weight to zero and completely remove them from the routing candidate set to prevent any requests from being forwarded to dangerous nodes.

[0113] First, a state vector reflecting the real-time health status of the nodes is constructed. Specifically, the gateway periodically sends heartbeat probes to multiple candidate underlying heterogeneous computing power nodes to obtain the real-time response latency and the number of currently active concurrent connections of each node, and constructs a state vector describing its real-time running status accordingly.

[0114] For example, in this embodiment, the target cluster Real-time state vector It can be represented as follows:

[0115]

[0116] in, For nodes that are periodically collected by heartbeat probes exist Real-time response latency at any given moment; For nodes The current number of active concurrent connections reflects the current load saturation level. Understandably, these two metrics quantify the health status of a node from different perspectives: latency reflects the node's processing speed and is a direct reflection of its responsiveness; concurrency reflects the node's resource utilization and is an indirect reflection of its load margin.

[0117] Secondly, define a veto step penalty function; after obtaining the real-time state vector, a step penalty function as the pre-multiplier of the routing weight needs to be defined, which is applied when the node response delay exceeds a preset hard threshold. When this happens, the function value immediately jumps from 1 to 0, causing the routing weight to return to zero.

[0118] For example, in this embodiment, for a node Step penalty function It can be represented as follows:

[0119]

[0120] in, This function sets the maximum tolerable response latency threshold for system administrators based on service quality requirements. The design logic is as follows: exceeding the latency limit indicates that a node has entered an unavailable state. At this point, any form of weight decay is too mild; a hard cutoff at the millisecond level must be achieved through mathematical nullification. This is the core mechanism for solving the routing rigidity problem. Figure 7 The figure shows a comparison curve between the step penalty function in this embodiment and the traditional linear decay function.

[0121] Furthermore, based on the veto guarantee, a nonlinear denominator reflecting the overall resistance of nodes is constructed, and a tenant permission multiplier is added to obtain the dynamic routing weight. This is achieved by applying weights to the real-time state vector. The norm is used to construct the denominator that reflects the overall resistance of the nodes, so that the routing weight exhibits a non-linear and rapid decreasing characteristic as the node load increases.

[0122] For example, in this embodiment, for a node Dynamic routing weights It can be calculated using the following formula:

[0123]

[0124] in, To adjust the weight vector, and Used respectively for balancing delay and concurrency The relative contribution to overall drag can be increased when it is desirable for routing decisions to be more sensitive to response delay. When it is desired that routing decisions are more sensitive to concurrency saturation, the value can be increased. ; This is the Hadamard element-wise product, used to apply weights to each dimension of the state vector before participating in norm calculation; for Norm, by adjusting The value of can control the sensitivity of the weight to the peak dimension. The larger the norm, the closer it is to taking the maximum component, making the deterioration of any dimension dominate the overall resistance; It is a very small positive number, used to prevent numerical singularities when the denominator is zero; As a molecule, it reflects the strategy that high-privilege tenants can obtain priority scheduling under the same cluster conditions.

[0125] It should be noted that the nonlinear denominator means that as latency or concurrency increases, the routing weight decreases at a superlinear rate. This allows the system to quickly and sensitively redirect request traffic away from heavily loaded nodes, avoiding the problem of traffic still pouring onto nodes that have clearly deteriorated but are still decreasing linearly.

[0126] Finally, the gateway selects the node with the highest overall routing weight as the forwarding target for the current request.

[0127] For example, in this embodiment, the finally selected heterogeneous target node It can be determined by the following formula:

[0128]

[0129] in, Given the set of all online heterogeneous candidate clusters, it is understandable that... Under this protection, the weights of all nodes with excessive latency have been set to zero. The operation will automatically select the optimal route target with the lowest overall cost from the nodes that are still in a healthy state, thus realizing a complete closed loop from health status perception to optimal node selection. Figure 8 The diagram shows a comparison of traffic transfer speeds for different routing strategies when node degradation occurs in this embodiment. In the diagram, (a) represents the traffic transfer speed of the traditional round-robin method; (b) represents the traffic transfer speed of the static weight method; and (c) represents the dynamic routing weight of this scheme. These three sub-graphs share the same X-axis to represent time slices.

[0130] Selected target heterogeneous computing power cluster Subsequently, the gateway uses the real physical authentication key obtained in step 2 and temporarily stored in the protected memory area to directly encapsulate it into the request header field (e.g., Authorization: Bearer) via the underlying network socket. <key>In this process, a TLS handshake is completed with the target heterogeneous computing cluster, and a logically isolated channel is established to carry out the request. Understandably, since the same gateway serves a large number of tenants at the same time, this logically isolated channel may reuse the same TCP connection pool at the physical layer, but at the logical layer, by binding a virtual identity token and request context separately for each request, it ensures that the request data, response data and audit metadata of different tenants are completely isolated within the gateway.

[0131] Step 5: During the process of the target heterogeneous computing power cluster returning streaming response data through the logical isolation channel, perform real-time computing power audit based on deterministic state machine. When the accumulated computing power consumption is detected to exceed the estimated safety quota level, trigger the economic circuit breaker mechanism and forcibly release the network connection of the logical isolation channel at the gateway layer. Otherwise, output standardized security audit logs after the transmission is completed.

[0132] Among them, streaming response data refers to the sequence of response content that the target heterogeneous computing power cluster returns to the gateway in the form of discrete data frames through streaming protocols such as SSE (Server-SentEvents) or chunked transmission encoding.

[0133] Real-time computational auditing based on deterministic state machines refers to the gateway converting each frame of streaming data into a standardized quota consumption amount and accumulating it in the step accumulator in real time when the data arrives. At the same time, based on the comparison between the accumulated value and the estimated security quota level, the gateway triggers a deterministic transition between the forwarding and circuit breaker discarding states.

[0134] In this embodiment, in order to monitor and control resource consumption in real time during the streaming output of the AI ​​model, and to prevent post-request deductions from causing the platform to be completely exposed to excessive resource consumption during the request process, real-time computational auditing based on deterministic state machines may specifically include:

[0135] Determining the target node in dynamic routing After forwarding the request, the final and most complex challenge facing the model is how to monitor and control resource consumption in real time during the AI ​​model streaming output (SSE) process. The streaming output of large AI models has significant uncertainty; a single request may continuously generate data over several seconds to several minutes. If the platform relies on post-request billing, it will be completely exposed to the risk of excessive resource consumption during the request process.

[0136] Therefore, this embodiment establishes a real-time economic circuit breaker model for a streaming state machine. The streaming output process can be modeled as a discrete frame sequence. When each frame of data arrives, the gateway converts the number of valid computing units contained in that frame into a normalized quota consumption amount and accumulates it in real time into a step accumulator. When the value of the accumulator exceeds the value pre-calculated in step 3... When the water level is reached, the state machine immediately switches from the forwarding state to the circuit breaker discard state, forcibly terminating the current streaming output.

[0137] First, an interception buffer and a step accumulator are initialized at the gateway layer. The gateway allocates a separate interception buffer for this request to temporarily store data frames returned by the downstream cluster; at the same time, a step accumulator is initialized. Its initial value is zero, used to accumulate and record the normalized quota consumption generated by this request. The gateway then listens for streaming data frames returned by the target heterogeneous computing cluster through the logical isolation channel.

[0138] Secondly, each data frame is deterministically segmented using a preset word segmentation algorithm, and this segmentation is added to the step accumulator to obtain the current cumulative consumption. The state of the gateway step accumulator can be updated in real time using the following recursive formula:

[0139]

[0140] in, The sequence number of the currently received streaming data frame; For the first The number of valid computational units contained in the frame data (usually the number of output tokens, specifically obtained by a preset word segmentation algorithm to perform deterministic segmentation and statistical analysis of the frame content); For the selected target cluster The output unit price; For the global benchmark exchange rate constant defined in step 3, ensure that the accumulated amount is consistent with... They can be directly compared under the same dimensions. Figure 9 This diagram illustrates a comparison of the excess economic loss exposure between real-time circuit breaking and post-charge in the flow state machine in this embodiment.

[0141] Understandably, the physical meaning of this recursive formula is: to perform precise frame-by-frame integration on discrete streaming data packets, transforming the economic consumption of the streaming output into a monotonically increasing curve over time, and tracking its relative position to the budget ceiling in real time. The reason for using a deterministic segmentation algorithm (rather than probabilistic or approximate estimation methods) is to ensure that the results of repeated calculations of the same response content at different times and on different gateway instances are completely consistent, thereby guaranteeing the reproducibility of the audit process and the reconciliation of accounts in case of settlement disputes.

[0142] Next, a hardware-level comparator is used to perform high-frequency comparisons between the current accumulated consumption and the estimated safety allowance level, and this comparison drives the state transitions of the state machine. The state transition functions of the state machine are defined as follows:

[0143]

[0144] It should be noted that in the above formula The steps in step 3 for the selected cluster were directly reused. The calculated standardized estimated safety quota level achieves a precise closure between the budget estimation stage and the real-time consumption monitoring stage. When First breakthrough When the upper bound is reached, the state machine immediately issues a HALT_AND_DROP instruction, which includes: discarding any remaining unforwarded data frames in the buffer pool, forcibly releasing the network connection of the logically isolated channel at the gateway layer (e.g., actively terminating the downstream connection by sending a TCPRST message), and returning an exception status code indicating quota exhaustion to the tenant side; conversely, when the streaming ends... If the condition is always true, the state machine will remain in the FORWARD state, and this request will be completed as a normal call, proceeding to the subsequent standardized security audit log output stage.

[0145] Understandably, the above mechanism does not allow any form of excess data to continue to be processed or included in the billing, thereby deterministically preventing excess economic losses in streaming scenarios.

[0146] Step 6: Output standardized security audit logs.

[0147] This embodiment also includes a further refined process for outputting standardized security audit logs. Standardized security audit logs refer to persistent data entries formed by structurally recording key events, resource consumption, and execution results of the current call using a pre-defined set of fields and semantic specifications. Specifically, this step includes:

[0148] First, a probe module is deployed in bypass mode at the gateway layer to capture non-standard operational status messages returned by the target heterogeneous computing power cluster in real time. Non-standard operational status messages refer to auxiliary information returned by the downstream heterogeneous computing power cluster during the response process, in addition to the mainstream content; for example, vendor-specific status codes, authentication failure prompts, rate limiting prompts, internal error codes, and the actual number of settlement tokens carried at the end of the model response. Since the message formats and status code semantics used by different heterogeneous computing power cluster vendors vary, without standardization, a consistent audit approach cannot be established at the platform layer.

[0149] Secondly, based on the preset protocol parsing rule tree, audit metadata is extracted from non-standard operating status messages. Specific extracted fields include: a unique request identifier (used to associate this request across systems), millisecond-level consumption time (i.e., the actual wall clock time from when this request enters the gateway to when the streaming transmission ends or the circuit breaker is triggered), the actual number of settlement calculation units (i.e., the number of real settlement tokens or bytes reported by the downstream cluster), and vendor-specific status codes.

[0150] Secondly, a hard-coded mapping table is used to convert vendor-specific status codes into globally unified standard response status codes. This mapping table records the correspondence between the original status codes of different heterogeneous computing power cluster vendors and the platform's standard status codes in key-value format; for example, various quota shortage errors from different vendors can be uniformly mapped to the standard status code 'QUOTA_EXHAUSTED', and various authentication failure errors from different vendors can be uniformly mapped to the standard status code 'AUTH_FAILED', thereby eliminating vendor differences at the auditing level.

[0151] Finally, the extracted and mapped fields are combined into a standardized security audit metadata structure and stored in the audit database. This structure is defined with a fixed schema, facilitating subsequent large-scale queries, statistics, and anomaly retrieval based on SQL or columnar storage engines.

[0152] Step 7: Detection of anomalous behavior baselines based on standardized security audit metadata.

[0153] In this embodiment, in order to achieve accurate perception of cross-hardware behavior anomalies in heterogeneous clusters in streaming output scenarios, and to accurately identify malicious probing attacks where the consumption is not excessive but the behavior pattern is severely abnormal (e.g., attackers continuously sniff parameters of the model interface in a low-frequency, long-duration manner), the abnormal behavior baseline detection specifically includes:

[0154] The economic circuit breaker mechanism in step 5 alone cannot identify malicious probing attacks that, while not exceeding the consumption limit, exhibit severely abnormal behavior patterns. Furthermore, the baseline response times for heterogeneous models of different sizes under normal requests vary greatly; the normal response time of a small model with seven billion parameters can differ by an order of magnitude from that of a large model with hundreds of billions of parameters. Therefore, it is necessary to establish a dynamic baseline independently for each tenant-cluster pair, rather than relying on a globally fixed threshold.

[0155] Therefore, this embodiment introduces the Exponential Moving Average (EMA) algorithm for tokens. In a specific cluster Time sequence of each request on Dynamically estimate the mean and variance. The core advantage of the EMA algorithm is that it assigns higher weights to the most recent observations, enabling the baseline to quickly adapt to normal behavioral drift. At the same time, its calculation only requires maintaining two scalars (mean and variance), resulting in extremely low storage and computational overhead. It is particularly suitable for continuous updates with extremely low latency in high-concurrency gateway scenarios.

[0156] First, extract the millisecond-level consumption time and request interval time series of the same virtual identity token within a historical time period from the audit database. The standardized security audit metadata generated in step 6 uses a composite index of virtual identity token and cluster identifier, which can retrieve the historical time sequence of the token on a specified cluster in constant time complexity.

[0157] Secondly, the dynamic baseline average time is calculated for this virtual identity token based on the EMA algorithm. and variance The specific update formula is as follows:

[0158]

[0159]

[0160] in, For the first The actual time consumed in milliseconds for each request; The EMA smoothing coefficient (typically between 0.1 and 0.3) is used to control the rate at which the baseline learns from the latest observations and the rate at which it forgets historical data. The larger the baseline, the more sensitive it is to new data, making it suitable for scenarios where behavioral patterns change rapidly; The smaller the value, the more stable the baseline and the stronger the noise immunity.

[0161] It should be noted that the variance update formula uses... Instead This involves calculating the residuals from the mean of the previous step. This is a standard online estimation paradigm that avoids circular dependencies and ensures the numerical stability of the variance estimate.

[0162] Furthermore, the classic 3-Sigma rule in statistics is used to define a dynamic anomaly threshold, and the judgment of time-consuming anomalies is logically combined with high-frequency packet sending behavior. When the time of a request exceeds the dynamic mean plus three times the dynamic standard deviation, the time-consuming behavior of that request is considered to be in the statistically significant anomaly range (corresponding to an extreme event with a probability of less than 0.3% under a normal distribution). At the same time, in order to further distinguish between occasional high latency caused by network jitter and genuine malicious behavior, the judgment of time-consuming anomalies is logically combined with high-frequency packet sending behavior. Only when both conditions are met simultaneously will a behavior anomaly alarm be triggered.

[0163] For example, in this embodiment, a Boolean decision function that triggers the baseline of abnormal behavior is used. It can be represented as follows:

[0164]

[0165] in, The dynamic anomaly threshold for the Gaussian distribution confidence interval is derived entirely from the historical behavior statistics of the token on this specific heterogeneous cluster, and has strong individual adaptability. The threshold for determining the rate of high-frequency packet transmission can be combined with the real-time attenuation rate calculated in step 2. Direct quote.

[0166] It should be noted that the logic and design of the two Boolean conditions enable the system to accurately distinguish between the following two scenarios: 1. When only the former is true (abnormal latency but normal frequency), it tends to indicate network jitter or fluctuations in the model itself, and no alarm is triggered; 2. When both are true (abnormal latency and high-frequency packet sending), it is judged with extremely high confidence as malicious probing or attack behavior, triggering an alarm and initiating circuit breaker handling. Understandably, the above judgment conditions can be extended to include M consecutive calls with millisecond-level latency all greater than [a certain value]. By introducing the constraint of continuous observation, the false alarm rate caused by sporadic jitter can be further reduced.

[0167] Finally, after determining the baseline of the triggered abnormal behavior, the isolation command is automatically executed. Specifically, the gateway forcibly revokes all real physical authentication key mapping relationships of the virtual identity token in the isolation mapping dictionary, that is, removes the composite data object corresponding to the virtual identity token from the dictionary or marks it as invalid, so that any subsequent call requests initiated by the token cannot complete the decoupling mapping and are directly intercepted by the gateway in step 2. Figure 10 A comparison chart of the time-consuming adaptive capabilities of the dynamic baseline and the globally fixed threshold cross-heterogeneous model in this embodiment is shown. Figure 11 The diagram shows the false alarm suppression effect of the Boolean decision function in this embodiment.

[0168] It should be noted that since the dynamic baseline of the EMA algorithm is maintained independently at the token-cluster pair granularity, the time consumption baseline for a small model with seven billion parameters is completely independent of the time consumption baseline for a large model with hundreds of billions of parameters. The system will not misjudge the normal latency of the large model based on the criteria for judging the small model. This fundamentally solves the problem of inconsistent audit baselines across heterogeneous hardware and makes up for the technical deficiency of existing audit log systems that can only check accounts after the fact and cannot intercept them in real time.

[0169] Step 8: Adaptive control mechanisms for extreme network partitioning or complete upstream cluster crash scenarios.

[0170] In this embodiment, in order to deal with extreme scenarios where all available heterogeneous computing power clusters are unable to establish logically isolated channels (e.g., wide area network partitioning, upstream vendor failures, large-scale DNS pollution, etc.), and to prevent the gateway from exhausting the suspended connection resources in memory due to continuous retrying of connection establishment in such scenarios, this embodiment also provides a local escape and degradation mechanism.

[0171] Specifically, when the call request in step 4 attempts to establish a logically isolated channel to all available alternative heterogeneous computing power clusters, and the timeout number of consecutive connection attempts exceeds the preset security tolerance, At that time, the gateway immediately triggers the local escape and degradation logic.

[0172] The specific execution steps of the local escape degradation logic include:

[0173] First, terminate the interaction with external physical computing power, that is, cancel all ongoing connection retry plans and mark the call request as entering a downgraded state;

[0174] Secondly, the original request path is blocked, that is, the gateway memory buffer and file descriptor already occupied by the request are released to avoid further resource consumption;

[0175] Next, the pre-loaded deterministic rule engine degradation configuration is extracted from the gateway's local non-volatile storage (e.g., gateway local SSD or Flash). This degradation configuration defines, in the form of a rule table, the fixed response content that should be returned to the tenant side under various degradation scenarios;

[0176] Finally, a formatted static degradation response packet containing cluster circuit breaker warning information is sent directly to the tenant side. For example, the HTTP status code '503 Service Unavailable' is returned, along with a response body of '{"code":"CLUSTER_CIRCUIT_BROKEN","message":"All downstream computing clusters are temporarily unavailable. Please try again later"}'.

[0177] This enables a deterministic, externally independent approach to respond quickly to extreme scenarios, ensuring that gateway memory resources are not exhausted by suspended connections in high-concurrency congestion environments, and protecting the availability of the gateway itself as the security boundary of the entire platform from collapsing due to downstream failures. Figure 12 The diagram illustrates the protection effect against extreme network partitions in this embodiment.

[0178] Step 9: Memory boundary isolation characteristics of the real physical authentication key.

[0179] Finally, in this embodiment, a strict memory boundary isolation scheme is provided to ensure that the real physical authentication key cannot be reverse-parsed by the end user in any form throughout its entire lifecycle within the gateway.

[0180] Specifically, after the virtual identity token is parsed in step 2, the mapping operation is performed only in the gateway's protected memory area (e.g., a separate memory segment marked PROT_READ via the mprotect system call, or a cryptographic enclave built on a hardware trusted execution environment such as IntelSGX / ARMTrustZone). The real physical authentication key is written to a read-only secure register that is invisible to external requesters throughout its lifetime. This register is only visible to the key reading routine of the underlying network socket and is not visible to the upper-layer business logic and debugging interface.

[0181] When constructing the actual physical request header sent to the target heterogeneous computing power cluster, the underlying network socket directly extracts the actual physical authentication key from the read-only security register via privileged system calls for encapsulation. This encapsulation operation is completed in kernel mode, avoiding the creation of addressable copies of the plaintext key in user-space memory. This design ensures that even if end users attempt to extract the actual physical authentication key through packet capture, parameter callbacks, or memory dumps of the gateway's service processes, they cannot reverse engineer it, thus guaranteeing the authentication security of the platform's computing power assets at the physical hardware level.

[0182] Example 2

[0183] This embodiment discloses a heterogeneous computing power cluster intensive security management and control system.

[0184] Specifically, the heterogeneous computing power cluster centralized security management system can be integrated into electronic devices, such as terminals, servers, and gateway devices. Terminals can be workstations, laptops, or personal computers; servers can be single physical servers, distributed server clusters of multiple servers, or virtualized gateway instances deployed in public, private, or hybrid cloud environments. When the electronic device is running, it can implement the heterogeneous computing power cluster centralized security management method as described in Embodiment 1 of this application.

[0185] In this embodiment, the heterogeneous computing power cluster centralized security management system can also be integrated into multiple electronic devices. For example, the heterogeneous computing power cluster centralized security management system can be integrated into multiple front-end proxy gateway servers, and the heterogeneous computing power cluster centralized security management method of Embodiment 1 of this application can be implemented by multiple front-end proxy gateway servers working together.

[0186] In this embodiment, the server can also be implemented as an edge gateway device.

[0187] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.< / key>

Claims

1. A method for intensive security management and control of heterogeneous computing power clusters, characterized in that, The intensive security management and control method includes: Obtain a call request for the first heterogeneous computing power cluster. The call request includes a first credential and request payload data. The first credential does not carry authentication information for the first heterogeneous computing power cluster. Based on the first credential query, a preset mapping data structure is obtained to obtain a second credential corresponding to the first credential and a feature profile bound to the first credential. The second credential is used to complete authentication with the first heterogeneous computing power cluster. The feature profile includes at least dynamic features that reflect the real-time consumption rate of the first credential. Cost normalization across heterogeneous environments is performed on the request load data to determine the estimated quota level of the call request, and pre-verification is performed based on the estimated quota level and the feature profile. When the pre-verification is passed, the second credential is used to establish an isolation channel with the first heterogeneous computing power cluster to forward the call request; During the process of the first heterogeneous computing power cluster returning streaming response data through the isolation channel, the cumulative consumption of the call request generated is accumulated in real time, with the same dimension as the estimated quota level. When the cumulative consumption exceeds the estimated quota level, forwarding is stopped and the isolation channel is released; otherwise, a standardized audit log is output after the transmission is completed.

2. The method for centralized security management of heterogeneous computing power clusters according to claim 1, characterized in that, The step of obtaining the call request for the first heterogeneous computing power cluster includes: The reverse proxy component completes the transport layer security handshake and protocol decapsulation for the call request; After the call request has been decapsulated, the request header field and request body are separated in a layered manner to extract the first credential from the request header field and the request payload data from the request body. The first credential and the request payload data are respectively written into different data areas in the memory of the processing device; The mapping data structure is a key-value data structure that resides in the high-speed memory of the processing device. Its key is the hash digest of the first credential, and its value is a composite data object that includes the second credential, permission level, global total quota limit, and historical behavior feature fingerprint.

3. The method for intensive security management and control of heterogeneous computing power clusters according to claim 1, characterized in that, Based on the first voucher query, a preset mapping data structure is obtained to obtain a second voucher corresponding to the first voucher and a feature profile bound to the first voucher, including: Perform an irreversible hash operation on the first credential to obtain the mapping query key; The composite data object is obtained by performing a constant-time lookup operation in the mapped data structure using the mapped query key. The second credential in the composite data object is extracted separately and temporarily stored in the protected memory area.

4. The method for intensive security management and control of heterogeneous computing power clusters according to claim 3, characterized in that, The second credential is written to a read-only security register that is not visible to external requesters during its lifetime; During the process of establishing an isolated channel with the first heterogeneous computing power cluster using the second credential, the system's underlying network socket directly extracts the second credential from the read-only security register through privileged system calls and encapsulates the request header fields. The privileged system call completes the encapsulation in kernel mode and does not generate an addressable copy of the second credential in user-mode memory; The feature profile also includes: The static features reflect the inherent attributes of the first credential, including at least one of the following: permission level, global total quota limit, and historical risk fingerprint score; the dynamic features also include the remaining resource quota level.

5. The method for intensive security management and control of heterogeneous computing power clusters according to claim 4, characterized in that, Before establishing an isolated channel with the first heterogeneous computing power cluster using the second credential, the method further includes: Real-time response latency and active concurrent connection count are periodically obtained for each candidate computing power cluster in the candidate computing power cluster set, and a real-time state vector is constructed for each candidate computing power cluster. A step penalty factor is determined for each candidate computing power cluster. The step penalty factor is 1 when the real-time response latency of the candidate computing power cluster does not exceed a preset response latency threshold and is 0 when it exceeds the response latency threshold. Using the permission level in the feature profile as the numerator, the comprehensive resistance value obtained by applying a weighted nonlinear norm to the real-time state vector as the denominator, and the step penalty factor as the pre-multiplier, the routing weight of each candidate computing power cluster is obtained. The candidate computing power cluster with the largest routing weight is selected as the first heterogeneous computing power cluster.

6. The method for centralized security management of heterogeneous computing power clusters according to claim 1, characterized in that, Performing cost normalization across heterogeneous environments on the request load data to determine the estimated quota level for the call request includes: The input length is extracted from the request load data, and an adaptive output length estimate is obtained based on the historical input and output length statistics corresponding to the first voucher. The adaptive output length estimate does not exceed the maximum output length limit corresponding to the first voucher. Based on the input length, the estimated adaptive output length, and the input / output billing unit price matrix of the first heterogeneous computing power cluster, the estimated original cost of this request is obtained. The estimated original cost is converted into the dimensionless estimated quota level using the platform's global benchmark exchange rate constant; The step of performing pre-verification based on the estimated quota level and the feature profile includes: Determine whether the remaining resource quota level in the feature profile is sufficient to cover the estimated quota level, and obtain the first Boolean determination result; Determine whether the real-time consumption rate in the feature profile is lower than a preset safety rate threshold to obtain a second Boolean determination result; The pre-verification is deemed to have passed only if both the first Boolean result and the second Boolean result are true; otherwise, the pre-verification is deemed to have failed and the call request is intercepted.

7. The method for intensive security management and control of heterogeneous computing power clusters according to claim 6, characterized in that, The estimated quota level is also superimposed with a load premium term that is positively correlated with the real-time load congestion rate of the first heterogeneous computing power cluster. The higher the real-time load congestion rate of the first heterogeneous computing power cluster, the higher the estimated quota level. The real-time load congestion rate is obtained by aggregating the processor utilization and queue backlog of the first heterogeneous computing power cluster periodically collected. The security rate threshold is dynamically configured based on the permission level in the feature profile; the higher the permission level, the more lenient the corresponding security rate threshold.

8. The method for intensive security management and control of heterogeneous computing power clusters according to claim 1, characterized in that, The real-time accumulation of the cumulative consumption of the call requests, which is of the same dimension as the estimated quota level, includes: Initialize the interception buffer pool and the step accumulator with an initial value of zero for the aforementioned call request; Each data frame of the streaming response data is monitored, and the data frame is segmented using a preset deterministic segmentation algorithm to obtain the number of effective computing units in the data frame. Using the output billing unit price of the first heterogeneous computing power cluster and the platform's global benchmark exchange rate constant, the number of effective computing units is converted into a single-frame consumption amount with the same dimension as the estimated quota level, and accumulated to the step accumulator, with the current value of the step accumulator as the accumulated consumption amount; When the cumulative consumption exceeds the estimated quota level, forwarding is stopped and the isolation channel is released, including: A hardware-level comparator is used to compare the accumulated consumption with the estimated quota level, driving the state machine to make deterministic transitions between the forwarding state and the circuit breaker discard state. When the cumulative consumption exceeds the estimated quota level for the first time, the state machine jumps to the circuit breaker and discard state, discards the remaining unforwarded data frames in the interception buffer pool, actively terminates the isolation channel by sending a connection reset message, and returns an abnormal status code of quota exhaustion to the tenant side. When the cumulative consumption does not exceed the estimated quota level at the end of the streaming response data transmission, the state machine remains in the forwarding state and enters the output of the standardized audit log.

9. The method for intensive security management and control of heterogeneous computing power clusters according to claim 1, characterized in that, The intensive security management and control method also includes: When the number of consecutive connection establishment timeouts for each candidate computing power cluster in the candidate computing power cluster set exceeds the preset connection establishment timeout tolerance, the call request is marked as entering a downgrade, the ongoing connection retry is canceled, and the memory buffer and file descriptor occupied by the call request are released. The pre-loaded local degradation configuration is extracted from the local non-volatile memory of the processing device, and a static degradation response packet containing cluster circuit breaker warning information is sent to the tenant side according to the local degradation configuration.

10. A centralized security management and control system for heterogeneous computing power clusters, characterized in that, The centralized security management and control system includes: processor; The memory stores a computer program, which, when executed by a processor, implements the method for centralized security management of heterogeneous computing power clusters as described in any one of claims 1 to 9.

Images