Isolation security verification method of multi-tenant processor and related device

Abstract

The present disclosure provides a method for verifying the isolation security of a multi-tenant processor and related equipment. The method comprises: in response to a first tenant executing an inference task, obtaining operation characteristic data of an operation behavior of a second tenant on a processor hardware resource used by the first tenant, and obtaining inference characteristic data of the first tenant; and performing data correlation analysis on the operation characteristic data and the inference characteristic data to generate a verification result. According to the embodiments of the present disclosure, a verification result indicating the effectiveness of the isolation between the multi-tenants under a preset isolation strategy can be obtained.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to an isolation security verification method and related equipment for a multi-tenant processor. Background Technology

[0002] With the rapid development of large language model (LLM) inference services and computing power sharing platforms, the deployment method of multiple tenants sharing the same processor to execute inference tasks has become widely used. This resource-sharing deployment mode can significantly improve the utilization efficiency of hardware computing resources.

[0003] In multi-tenant processor-shared operation scenarios, relevant technologies typically employ isolation strategies to isolate the business resources of different tenants, thereby reducing the risk of privacy data leakage between tenants. However, there is currently a lack of systematic verification methods for the actual protective effectiveness of isolation strategies, making it difficult to effectively verify the effectiveness of isolation between business resources of different tenants during actual inference business operations, resulting in significant uncertainty in security control. Summary of the Invention

[0004] This disclosure provides a method and apparatus for isolating and verifying security of a multi-tenant processor, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] In a first aspect, this disclosure provides an isolation security verification method for a multi-tenant processor. The method includes: in response to a first tenant executing an inference task, acquiring operation feature data of an operation performed by a second tenant on processor hardware resources used by the first tenant, and acquiring inference feature data of the first tenant; wherein the first tenant and the second tenant are different tenants isolated within the processor based on a preset isolation strategy; the operation feature data represents the operation behavior; the inference feature data represents the activity state of the first tenant in the inference task; performing data correlation analysis on the operation feature data and the inference feature data to generate a verification result, wherein the data correlation analysis analyzes the data change trends and / or numerical feature correspondences between the operation feature data and the inference feature data; the verification result indicates the isolation effectiveness between multiple tenants under the preset isolation strategy.

[0006] Secondly, this disclosure provides an isolation security verification device for a multi-tenant processor. The device includes: an acquisition module, configured to, in response to a first tenant executing an inference task, acquire operation feature data of an operation performed by a second tenant on the processor hardware resources used by the first tenant, and acquire inference feature data of the first tenant; wherein the first tenant and the second tenant are different tenants isolated in the processor based on a preset isolation strategy; the operation feature data represents the operation behavior; the inference feature data represents the activity state of the first tenant in the inference task; and a verification module, configured to perform data correlation analysis on the operation feature data and the inference feature data to generate a verification result, wherein the data correlation analysis is used to analyze the data change trends and / or numerical feature correspondences between the operation feature data and the inference feature data; and the verification result represents the isolation effectiveness between multiple tenants under the preset isolation strategy.

[0007] Thirdly, this disclosure provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the above-described isolation security verification method.

[0008] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described isolation security verification method.

[0009] Fifthly, this disclosure provides a computer program product that includes computer-readable code or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described isolation security verification method.

[0010] The embodiments provided in this disclosure target a first tenant and a second tenant isolated in a processor based on a preset isolation strategy. In response to the first tenant executing an inference task, the second tenant is synchronously controlled to perform operations on the processor hardware resources used by the first tenant. Operation feature data representing the operation behavior and inference feature data representing the activity state of the first tenant in the inference task are obtained. By performing data correlation analysis on the operation feature data and the inference feature data, the data change trends and / or numerical feature correspondence between the operation feature data and the inference feature data are accurately quantified and mined. This enables the accurate capture of the inherent coupling relationship between implicit correlation behaviors such as processor hardware resource contention, timing fluctuations, residual video memory data, and data interconnection interference and tenant inference activities in a multi-tenant shared processor scenario. This generates a verification result that can intuitively represent the isolation effectiveness between multiple tenants under the preset isolation strategy. Based on the verification result, the preset isolation strategy is updated to strengthen the isolation effect, thereby solving the problem of privacy data leakage between different tenants.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the description of detailed exemplary embodiments with reference to the accompanying drawings, which are described below.

[0013] Figure 1 This is a flowchart of an isolation security verification method provided in an embodiment of the present disclosure.

[0014] Figure 2 This is a first flowchart for obtaining feature data according to an embodiment of the present disclosure.

[0015] Figure 3 This is a second flowchart for obtaining feature data according to an embodiment of the present disclosure.

[0016] Figure 4 A third flowchart for obtaining feature data provided in an embodiment of this disclosure.

[0017] Figure 5 The fourth flowchart for obtaining feature data is provided in the embodiments of this disclosure.

[0018] Figure 6 The fifth flowchart for obtaining feature data is provided in the embodiments of this disclosure.

[0019] Figure 7 This is a block diagram of an isolation security verification device provided in an embodiment of the present disclosure.

[0020] Figure 8 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation

[0021] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0022] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0023] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0024] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0025] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0026] In large-scale multi-tenant inference applications, multi-tenant shared processor inference has become the mainstream deployment mode in order to improve the utilization efficiency of processors, especially graphics processing units (GPUs). Multi-tenant shared processor inference refers to the concurrent execution of inference requests from multiple users or tenants on the same physical processor (e.g., GPU), sharing computing resources and cache hierarchy.

[0027] In developing this disclosure, the applicant discovered that in multi-tenant processor inference environments, multi-tenancy can typically be deployed on processors based on isolation strategies such as batched serving, multi-instance GPU (MIG) isolation, time-sharing, and virtualized GPU (vGPU) allocation to achieve data isolation between different tenants and reduce the risk of privacy data leakage. Specifically, the batched serving isolation strategy is generally used to merge multiple inference requests within the same GPU instance for batch processing, commonly seen in continuous batch processing scenarios such as vLLM (very-large-language-model inference serving engine) and TGI (Text Generation Inference); the MIG isolation strategy typically involves hardware partitioning and isolation within the same physical GPU, often used in cloud GPU services; the time-sharing isolation strategy typically involves multiple containers or processes alternately using the GPU, commonly seen in Kubernetes GPU sharing scenarios; and the vGPU isolation strategy typically allocates computing resources in a virtualized form, often applied in VMware application environments.

[0028] In the process of realizing this disclosure, the applicant discovered that with the widespread use of multi-tenant processor inference environments, various information leakage pathways have emerged in related technologies that take advantage of the shared characteristics of processor hardware resources. These include data leakage caused by uncleaned hardware memory residues, as well as various side-channel information leakage problems such as cache resource contention, differences in inference runtime, and cross-device interconnection resource contention. Furthermore, there may be situations where malicious tenants preempt computing resources, causing a decrease in the inference service efficiency of other tenants.

[0029] While some verification methods exist in related technologies to address the aforementioned issues, they are often too one-sided. For example, the computing power side channel detection method usually focuses on verifying the integrity of kernel program operation and does not specifically verify scenarios such as information leakage between multi-tenants and mutual interference of resources.

[0030] In view of this, to address the problem in related technologies that lack a systematic verification method for the actual protective effect of isolation strategies in multi-tenant processor inference environments, making it difficult to effectively verify the isolation effectiveness of business resources between different tenants during actual inference business operations, this disclosure provides an isolation security verification method for multi-tenant processors, comprising: in response to a first tenant executing an inference task, acquiring operation feature data of the operation behavior performed by a second tenant on the processor hardware resources used by the first tenant, and acquiring inference feature data of the first tenant; wherein, the first tenant and the second tenant are different tenants isolated in the processor based on a preset isolation strategy; the operation feature data is used to represent the operation behavior; the inference feature data is used to represent the activity state of the first tenant in the inference task; performing data correlation analysis on the operation feature data and the inference feature data to generate a verification result, wherein the data correlation analysis is used to analyze the data change trend and / or numerical feature correspondence between the operation feature data and the inference feature data; the verification result indicates the isolation effectiveness between multiple tenants under the preset isolation strategy.

[0031] Based on the method provided in this disclosure, for a first tenant and a second tenant isolated in a processor according to a preset isolation strategy, in response to the first tenant executing an inference task, the second tenant is synchronously controlled to perform operations on the processor hardware resources used by the first tenant, and operation feature data representing the operation behavior and inference feature data representing the activity state of the first tenant in the inference task are obtained. By performing data correlation analysis on the operation feature data and the inference feature data, the data change trend and / or numerical feature correspondence between the operation feature data and the inference feature data are accurately quantified and mined, thereby accurately capturing the inherent coupling relationship between implicit correlation behaviors such as processor hardware resource contention, timing fluctuations, residual video memory data, and data interconnection interference and tenant inference activities in a multi-tenant shared processor scenario. This generates a verification result that can intuitively represent the isolation effectiveness between multiple tenants under the preset isolation strategy, and updates the preset isolation strategy based on the verification result to strengthen the isolation effect, thereby solving the problem of privacy data leakage between different tenants.

[0032] The isolation security verification method for a multi-tenant processor according to embodiments of this disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be an in-vehicle device, user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, wearable device, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Alternatively, the method can be executed by a server.

[0033] Figure 1 A flowchart illustrating an isolation security verification method for a multi-tenant processor provided in this disclosure. (Refer to...) Figure 1 The method may include the following steps S11-S12, which are described in detail below.

[0034] Step S11: In response to the first tenant performing an inference task, obtain the operation feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and obtain the inference feature data of the first tenant.

[0035] Among them, the first tenant and the second tenant are different tenants isolated in the processor based on a preset isolation strategy; the operation feature data is used to represent the operation behavior; the inference feature data is used to represent the activity status of the first tenant in the inference task.

[0036] In this embodiment of the disclosure, the processor hardware resources may be at least one of the following: video memory, caches at all levels, and data interconnect resources used for inter-card data communication that can be called by the tenant during the processor's inference task execution. This disclosure does not impose any special limitations on these resources.

[0037] It should be noted that the processor can be a graphics processing unit (GPU), a neural network processor, an AI accelerator card, an ASIC computing chip, etc. Unless otherwise specified, the following example will use a graphics processing unit (GPU) as the processor.

[0038] A tenant is a user that performs inference operations based on processor resources. Different tenants can divide the usage rights of processor hardware resources based on a preset isolation strategy.

[0039] In the embodiments disclosed herein, the preset isolation strategy may be one or more combinations of isolation strategies such as batch processing sharing, multi-instance GPU isolation, time-division multiplexing, and virtual graphics processing unit (vGPU), and this disclosure does not impose any special limitations on it.

[0040] This reasoning task can be a data computation or information processing business based on a large language model.

[0041] This operational characteristic data can be used to record behavioral information generated during the process of a second tenant accessing shared hardware resources.

[0042] This inference feature data can be used to record information such as the load status and runtime sequence during the first tenant's inference task execution.

[0043] Step S12: Perform data correlation analysis on the operational feature data and inference feature data to generate verification results.

[0044] Among them, data correlation analysis is used to analyze the data change trends and / or numerical feature correspondence between operational feature data and inference feature data; the verification results indicate the isolation effectiveness between multiple tenants under the preset isolation strategy.

[0045] The data correlation analysis can be an analysis and processing method that obtains the data change trends and / or the correspondence between numerical features of operational feature data and inference feature data based on relevant operational rules such as statistics and information theory. Based on the data correlation analysis, it can be determined whether the operation behavior of the second tenant on the hardware processor resources used by the first tenant during or after the first tenant performs the inference task can directly or indirectly obtain the privacy data of the first tenant, thereby obtaining a verification result that can intuitively determine the isolation effectiveness under the preset isolation strategy.

[0046] In some embodiments, the data correlation analysis may be performed by analyzing the correlation between the operational feature data and the inference feature data through numerical matching comparison; or by using mutual information calculation or by using classification matching determination.

[0047] By performing data correlation analysis on the operational feature data and the inference feature data to explore the mutual influence between tenant behaviors, the accuracy of isolation defect identification can be improved, thereby making the verification results more consistent with the actual shared operation status of hardware resources.

[0048] The verification results can be used to quantify the actual compliance level of resource allocation and data protection among different tenants.

[0049] That is, in this embodiment of the disclosure, for the first tenant and the second tenant isolated in the processor based on a preset isolation strategy, the inference feature data corresponding to the first tenant and the operation feature data corresponding to the second tenant are collected synchronously, and the operation feature data and inference feature data are analyzed and calculated to output a verification result that can objectively reflect the actual effect of resource isolation between multiple tenants.

[0050] As can be seen, based on the method provided in this disclosure embodiment, for a first tenant and a second tenant isolated in a processor according to a preset isolation strategy, in response to the first tenant executing an inference task, the second tenant is synchronously controlled to perform operations on the processor hardware resources used by the first tenant, and operation feature data representing the operation behavior and inference feature data representing the activity state of the first tenant in the inference task are obtained. By performing data correlation analysis on the operation feature data and the inference feature data, the data change trend and / or numerical feature correspondence between the operation feature data and the inference feature data can be accurately quantified and mined, thereby accurately capturing the inherent coupling relationship between implicit correlation behaviors such as processor hardware resource contention, timing fluctuations, residual video memory data, and data interconnection interference and tenant inference activities in a multi-tenant shared processor scenario. This generates a verification result that can intuitively represent the isolation effectiveness between multiple tenants under the preset isolation strategy, and updates the preset isolation strategy based on the verification result to strengthen the isolation effect, thereby solving the problem of privacy data leakage between different tenants.

[0051] The isolation security verification method according to the embodiments of this disclosure will now be described in detail.

[0052] In some embodiments, the data correlation analysis described in step S12 can be performed from at least one of the following detection dimensions: memory residue detection dimension, used to detect the risk of cross-tenant data residue leakage caused by un-zeroed memory resources after release; cache side-channel detection dimension, used to detect the risk of side-channel attacks that can steal tenant computing characteristics caused by tenant contention for processor cache resources; time side-channel detection dimension, used to detect the risk of stealing time-series information that infers tenant service types based on differences in inference task processing time; cross-tenant resource interference detection dimension, used to detect the risk of resource crowding out caused by tenants preempting hardware computing resources, leading to a decrease in the service performance of other tenants; and data interconnect side-channel detection dimension, used to detect the risk of leakage of cross-virtual machine and / or cross-tenant communication information caused by contention for multi-processor interconnect channels.

[0053] Specifically, as mentioned above, isolation strategies such as batch processing sharing, MIG hardware isolation, time-division multiplexing, and vGPU virtualization in related technologies often only provide idealized isolation and protection capabilities. However, their isolation effects are often difficult to quantify and verify accurately. For example, computing power side-channel detection methods usually focus on verifying the integrity of kernel program operation and do not specifically verify scenarios such as information leakage and resource interference between multi-tenants.

[0054] In the process of implementing this disclosure, the applicant discovered that inference frameworks such as vLLM and TGI typically do not actively clear GPU memory data when releasing KV Cache, which can easily lead to static data residue. Furthermore, since GPUs share L2 cache, cache line contention often occurs, which may induce cache side-channel attacks such as GhostWriter. At the same time, due to the inherent differences in runtime latency between inference tasks of different specifications, attackers may be able to reverse-engineer business privacy. High-load tenants are prone to preempting computing power and bandwidth resources, which can degrade the performance of other inference services. In addition, the sharing and reuse of NVLink and PCIe interconnect channels in multi-card networking can also create hidden leakage paths across VMs and tenants. For a long time, there has been a lack of unified and quantitative verification solutions for the above-mentioned risks.

[0055] Therefore, in this embodiment of the disclosure, data correlation analysis is carried out by analyzing the data change trend and numerical feature correspondence between the inference feature data of the first tenant and the operation feature data of the second tenant. Isolation effectiveness is verified by any one or more of the following five detection dimensions: memory residue detection dimension, cache side channel detection dimension, time side channel detection dimension, cross-tenant resource interference detection dimension, and data interconnection side channel detection dimension.

[0056] Specifically, this memory residue detection dimension can be used to detect the risk of cross-tenant data residue leakage caused by memory resources not being cleared after release.

[0057] In other words, in continuous batch inference scenarios such as vLLM and TGI, MIG hardware partition isolation, and vGPU virtualization shared scenarios, the video memory and KV cache memory regions may only be logically released without being forcibly zeroed, which may result in the retention of inference feature data of previous tenants. Therefore, this disclosure embodiment uses feature pattern matching and the ratio of residual data volume to quantify based on this detection dimension, accurately identifying privacy leakage vulnerabilities caused by static residual video memory, and verifying the actual isolation effect of video memory resources.

[0058] This cache side-channel probing dimension can be used to detect the risk of side-channel attacks that could steal tenant computing features, caused by tenant contention for processor cache resources.

[0059] In other words, under a multi-tenant shared GPU L2 cache architecture, even with isolation strategies such as MIG isolation and time-division multiplexing isolation for tenant isolation, the computational loads of different tenants can still compete for cache line resources, causing cache eviction and access latency fluctuations. Therefore, this disclosure embodiment quantifies the correlation between cache latency fluctuations and tenant inference activity patterns based on this detection dimension through information-theoretic mutual information calculation, and detects cache side-channel vulnerabilities that can be exploited by malicious tenants to steal each other's computational characteristics.

[0060] This time-side channel detection dimension can be used to detect the risk of theft of timing information that infers the tenant's service type based on the difference in inference task processing time.

[0061] In other words, inference tasks with different prompt lengths, batch sizes, and model types may have varying runtimes, and these timing differences cannot be naturally masked in software batch sharing and container time-division multiplexing scenarios. Therefore, this disclosure embodiment quantifies the inferability of timing information and accurately determines the degree of isolation failure of the time-side channel by probing the prediction matching degree of the tenant inference request type based on this probing dimension and the latency sequence.

[0062] This cross-tenant resource interference detection dimension can be used to detect the risk of resource crowding out caused by tenants preempting hardware computing resources, resulting in a decline in the business performance of other tenants.

[0063] In other words, during multi-tenant concurrent inference, resource contention arises regarding computing power, memory bandwidth, and scheduling time slots. High-load tenants can increase the inference latency of coexisting tenants, causing performance fluctuations. Therefore, this disclosure, based on this detection dimension, calculates a standardized resource interference index by comparing the exclusive baseline latency distribution with the measured coexistence latency distribution. This quantifies the computing power resource contention defects under different isolation strategies and verifies the effectiveness of resource scheduling isolation.

[0064] This data interconnect side channel detection dimension can be used to detect the risk of leakage of cross-virtual machine and / or cross-tenant communication information caused by multiprocessor interconnect channel contention.

[0065] In multi-card distributed inference scenarios, tenants in different isolated zones may share interconnection channels such as NVLink and PCIe. Bandwidth congestion and timing disturbances in cross-card data transmission can create hidden side channels. Therefore, this disclosure identifies the risk of leakage of interconnection resource information across tenants and virtual machines by analyzing the correlation between interconnection detection duration and tenant cross-card transmission activities based on this detection dimension.

[0066] As can be seen, the method provided by the embodiments of this disclosure, by using at least one of the above five detection dimensions, especially by combining multiple dimensions, can cover all shared hardware links such as video memory, cache, operation timing, computing power bandwidth and card interconnection, so that the method provided by the embodiments of this disclosure can both identify known side-channel attack vulnerabilities and capture unknown isolation failure risks.

[0067] Please refer to Figure 2 This is a first flowchart of the process for obtaining feature data provided in an embodiment of this disclosure. For example... Figure 2As shown, in some embodiments, data correlation analysis may include analysis from the perspective of residual memory detection; in this embodiment, processor hardware resources may include memory resources; the generation of verification results based on operational feature data and inference feature data in step S11 may include the following steps S21-S22.

[0068] Step S21: During the first tenant's inference task, control the first tenant to write target feature pattern data to the target memory region of the allocated video memory resources, and generate inference feature data based on the target feature pattern data.

[0069] Step S22: After the first tenant completes the inference task and releases the allocated video memory resources, the second tenant is controlled to perform a data reading operation on the target video memory region without performing a video memory initialization operation, and operation feature data is generated based on the data reading result.

[0070] Video memory resources can be memory areas within the GPU used for temporary storage of computational data. This target video memory area can be a segment of video memory space allocated to the first tenant during the execution of inference tasks.

[0071] That is, in some embodiments, the effectiveness of isolation can be determined from the detection dimension of whether there is residual memory under the adopted isolation strategy.

[0072] Specifically, the first tenant can write specific target feature pattern data during the execution of the inference task, and can generate the inference feature data based on the target feature pattern data.

[0073] In this embodiment, the target feature pattern data can be a uniquely identifiable feature pattern data written by the first tenant into the target memory area during the detection processing of this detection dimension. The target feature pattern data has the characteristics of being easy to identify and having a low probability of misjudgment.

[0074] The target feature pattern data can be, for example, at least one of 0xDEADBEEF, 0xCAFEBABE, 0xA5A5A5A5, and 0x5A5A5A5A.

[0075] For example, the target feature pattern data can be 0xDEADBEEF, which is a 32-bit hexadecimal number. It has a very low probability of appearing randomly in natural business scenarios and has high recognizability. It is usually difficult to generate naturally and randomly. Therefore, a certain amount of the target feature pattern data can be written to the target memory area of ​​the memory resources allocated to it by the first tenant during the execution of the inference task.

[0076] In some embodiments, the target feature pattern data can be generated by repeatedly writing at least one of the following data to the target video memory region: 0xDEADBEEF, 0xCAFEBABE, 0xA5A5A5A5, and 0x5A5A5A5A. For example, 0xDEADBEEF can be repeatedly written to the target video memory region; or, for another example, the five data types can be repeatedly written to the target video memory region in the order of 0xDEADBEEF, 0xCAFEBABE, 0xA5A5A5A5, and 0x5A5A5A5A.

[0077] In some embodiments, to avoid misjudgment caused by the second tenant mistakenly reading the target feature pattern data written to the target memory area in other rounds, when generating the target feature pattern data, the target feature pattern data can also be generated based on the current round identifier (test_round_id) used to represent the current verification round, the tenant identifier (tenant_A_id) of the first tenant, and the current timestamp, and the generated target feature pattern data can be repeatedly written to the target memory area.

[0078] That is, for example, the target feature pattern data can be obtained by: target feature pattern data = hash(test_round_id, tenant_A_id, timestamp), which means that the target feature pattern data is obtained by obtaining the hash value of the current round identifier, the tenant identifier of the first tenant, and the current timestamp; where hash() is a function to calculate the hash value. In actual implementation, the target feature pattern data can be obtained based on any hash algorithm, and this disclosure does not make any special restrictions on it.

[0079] For example, the target feature pattern data can also be obtained by directly concatenating the current round identifier, the tenant identifier of the first tenant, the current timestamp, and at least one of the following data: 0xDEADBEEF, 0xCAFEBABE, 0xA5A5A5A5, and 0x5A5A5A5A.

[0080] In practice, other methods can also be used to generate target feature pattern data that is easy to identify and reduces the probability of misjudgment. This disclosure does not impose any special limitations on this.

[0081] In practice, during the execution of an inference task, the first tenant can write a certain amount of target feature pattern data to the GPU target memory region allocated to it. After completing the inference task, the first tenant releases the target memory region, and inference feature data representing the first tenant's inference activity status can be generated based on the target feature pattern data. Subsequently, the second tenant requests GPU memory and reads the target memory region without initialization, and based on the data read results, generates operation feature data representing the second tenant's operational behavior towards the target memory region.

[0082] Subsequently, in this embodiment, the step of performing data correlation analysis on the operational feature data and the inference feature data to obtain the verification result may include: generating a first verification result by determining whether the operational feature data includes data that matches the target feature pattern data in the inference feature data; wherein, the first verification result represents the effectiveness of memory isolation under the preset isolation strategy.

[0083] That is, the effectiveness of memory isolation under the preset isolation strategy can be determined by determining whether the operation feature data containing the data reading results of the second tenant includes the target feature pattern data written by the first tenant into the target memory region during the execution of the inference task.

[0084] It is understandable that if the target feature pattern data is not included in the operation feature data, it can be said that the preset isolation strategy is effective; otherwise, it can usually be said that the preset isolation strategy has residual memory, which may lead to the leakage of tenant's privacy data.

[0085] That is, if tenant A represents the first tenant and tenant B represents the second tenant, the above process can be represented as follows: 1. Tenant A writes 0xDEADBEEF to video memory area 1 and then releases video memory area 1; 2. Tenant B requests video memory without performing initialization operations and directly reads data from video memory area 1; 3. If 0xDEADBEEF is read, it indicates that there is a video memory leak.

[0086] It should be noted that, in this implementation, in addition to determining the effectiveness of memory isolation, the ratio between the amount of target feature pattern data written by the first tenant and the amount of target feature pattern data read by the second tenant can be statistically determined to quantify the degree of memory residual leakage.

[0087] That is, in this embodiment, the inference feature data may further include a first data volume, which represents the amount of target feature pattern data written by the first tenant to the target memory area; the operation feature data may further include a second data volume, which represents the amount of target feature pattern data read by the second tenant from the target memory area.

[0088] The step of generating a first verification result by determining whether the operational feature data includes data that matches the target feature pattern data in the inference feature data may include: if the operational feature data includes the target feature pattern data, determining the ratio between the second data volume and the first data volume, and generating the first verification result based on the ratio, wherein the ratio is used to quantify the degree of residual leakage in the video memory.

[0089] For example, if the first tenant writes 1 gigabyte (GB) of target feature pattern data to the target memory region, and the second tenant reads 4 megabytes (MB) of the same target feature pattern data, then the number of leaked bytes can be determined to be 4MB; this ratio = 4MB / 1GB, and a first verification result that can be quantified is generated based on this ratio.

[0090] It should be noted that in actual implementation, for example, the detection processing based on this detection dimension can be used to detect the effectiveness of memory isolation in application scenarios such as memory release and reallocation within the same MIG instance under the MIG isolation policy, memory regions across MIG instances, memory status after the destruction of the GPU runtime context (CUDAContext), and memory status after the release of the vLLM / TGI KV Cache pool. This is to avoid the problem of privacy data leakage between tenants under this isolation policy and improve the security of tenant privacy data.

[0091] As can be seen, the first verification result generated based on the verification method for detecting video memory residue provided in this embodiment of the present disclosure, by using the feature data matching result as the judgment basis, is conducive to objectively reflecting the actual state of video memory isolation and facilitating staff to quickly identify information leakage risks at the video memory level. In addition, by using the data volume ratio to quantify the degree of video memory residue leakage, it is helpful to refine the evaluation criteria for video memory isolation effect, thereby enabling users to intuitively distinguish the severity of video memory security risks.

[0092] Please refer to Figure 3 This is a second flowchart of the process for obtaining feature data provided in an embodiment of this disclosure. For example... Figure 3 As shown, in some embodiments, data correlation analysis may include analysis from the cache side-channel probing dimension; processor hardware resources may also include processor cache resources, such as the L2 cache in the GPU.

[0093] In this embodiment, step S11, which involves obtaining the operation feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and obtaining the inference feature data of the first tenant, may include the following steps S31-S33.

[0094] Step S31: During the first tenant's inference task, control the first tenant to perform processor operation processing in a preset activity mode based on processor cache resources.

[0095] Step S32: During the first tenant's execution of processor operations, the second tenant is controlled to perform multiple probe operations on a preset memory region in the processor cache and obtain the access latency duration of each probe operation.

[0096] Step S33: Generate inference feature data based on the time information of the first tenant's processor operation; and generate operation feature data based on the obtained multiple access delay durations.

[0097] That is, in this embodiment of the disclosure, processor cache resources can be used as a detection carrier to determine the effectiveness of cache isolation under a preset isolation strategy from the detection dimension of cache side-channel. In this implementation, inference feature data of the first tenant and operation feature data of the second tenant can be generated based on operation timing and detection latency information. It should be noted that, in actual implementation, isolation security verification of this detection dimension can, for example, be used to detect GPU L2 cache contention-type side-channel problems exploited by GhostWriter attacks.

[0098] The preset activity mode can be at least one of the following: first tenant idle, performing small-scale matrix multiplication, large-scale matrix multiplication, first type of prompt word reasoning, and second type of prompt word reasoning. The first type of prompt word reasoning can be prompt word reasoning with a prompt word length greater than a first preset threshold, and the second type of prompt word reasoning can be prompt word reasoning with a prompt word length less than a second preset threshold. The first preset threshold is greater than the second preset threshold. The first preset threshold and the preset threshold can be set as needed, and this disclosure does not impose any special limitations on them.

[0099] This probing process can be a detection operation where a second tenant actively accesses a specified region of the cache and collects the hardware response time. For example, a second tenant can perform a Cache Probe operation to access a set of known memory addresses and accurately time the process.

[0100] The access latency refers to the time it takes for the hardware to receive the access instruction and complete the data retrieval when the second tenant performs a probe process, such as a Cache Probe operation.

[0101] This detection process can be represented, for example, by the following pseudocode.

[0102] for each cache_line in probe_set:

[0103] start = clock() / / Read the GPU timer value before accessing cache_line.

[0104] load(cache_line) / / Access cache_line.

[0105] end = clock() / / GPU timer value read after accessing cache_line.

[0106] latency[cache_line] = end – start / / The access latency corresponding to the current access to cache_line.

[0107] Here, cache_line, or cache line, can be a preset memory address used by the second tenant for probing. It can be a segment of GPU memory addresses, or it can be a set of memory locations associated with GPU cache resources, such as GPU L2 Cache mapping; latency[cache_line] can be dictionary-type data used to store the latency of a single access, which can be used to represent the time observation value obtained by the second tenant sampling the processor cache resources.

[0108] If a cache line is hit in the L2 cache, its corresponding access latency is usually low; if the cache line is evicted or contention occurs due to the default activity mode performed by the first tenant, such as performing a large matrix multiplication, the corresponding access latency will usually increase.

[0109] In this embodiment, the step of performing data correlation analysis on the operation feature data and inference feature data to obtain the verification result may include: determining the correspondence between each access delay duration and the preset activity mode based on the time information in the inference feature data; determining the first mutual information between the operation feature data and the inference feature data based on the correspondence; and generating a second verification result based on the first mutual information; wherein the second verification result represents the cache isolation effectiveness under the preset isolation strategy.

[0110] Specifically, in this implementation, for the operation feature data and the inference feature data, the computational activities performed by the first tenant in the inference activity within a certain time range can be determined by the time information and preset activity mode in the inference feature data. At the same time, the change in access latency when the second tenant performs probe processing within the same time range can be determined, thereby verifying the effectiveness of cache isolation under the preset isolation strategy, and determining whether there is cache contention, such as L2 cache contention side channel, which may lead to the leakage of the first tenant's computational features.

[0111] In practice, this first mutual information can be used to represent the amount of cached information leakage, which can be obtained by the following formula.

[0112] First mutual information = mutual_information(A_activity_pattern, B_latency_pattern).

[0113] Among them, mutual_information() is an information theory metric that can be used to measure the degree of correlation between two sets of data or two events. Generally, the higher the value, the stronger the correlation. In this embodiment of the disclosure, it can be used to indicate how much information about the activity mode of the first tenant is contained in the observation results of the access delay duration of the second tenant.

[0114] A_activity_pattern can be used to represent the activity mode of the first tenant. It can be at least one of the activity modes such as idle, performing small-scale matrix multiplication, large-scale matrix multiplication, first-type prompt word reasoning, and second-type prompt word reasoning. For example, the value 0 can be used to represent the idle activity mode, the value 1 to represent the small-scale matrix multiplication activity mode, the value 2 to represent the large-scale matrix multiplication activity mode, the value 3 to represent the first-type prompt word reasoning activity mode, and the value 4 to represent the second-type prompt word reasoning activity mode. Of course, in actual implementation, the activity mode and the values ​​of different activity modes can be set as needed, and this disclosure does not make any special limitations on this.

[0115] B_latency_pattern can represent a set of access latency durations obtained by the second tenant performing multiple probe processes. It is not the value of a single access latency duration, but a latency vector or duration sequence with a sufficient access latency duration. For example, it can be in the form of [access latency duration 1, access latency duration 2, ...].

[0116] Typically, if the inference activity, i.e. the activity pattern, of the first tenant may affect the processor's cache resources, such as the GPU L2 cache, then the access latency of the second tenant will vary with the inference activity of the first tenant.

[0117] For example, when the first tenant is in idle activity mode, the access latency for the second tenant might be 80 nanoseconds (ns), 82 nanoseconds, 79 nanoseconds, and 81 nanoseconds, respectively, i.e., B_latency_pattern1 = [80ns, 82ns, 79ns, 81ns]; while when the first tenant is in large-scale matrix multiplication activity mode, the access latency for the second tenant might be 140 nanoseconds (ns), 155 nanoseconds, 148 nanoseconds, and 160 nanoseconds, respectively, i.e., B_latency_pattern2 = [140ns, 155ns, 148ns, 160ns]. This shows that although the second tenant does not directly access the first tenant's data, the latency variation can be used to infer whether the first tenant is performing high-intensity computation.

[0118] In this embodiment of the disclosure, when the first mutual information obtained in this way is 0, it can be said that the delayed access duration of the second tenant is unrelated to the inference activity of the first tenant, that is, the cache isolation is relatively effective; otherwise, the larger the value, the worse the cache isolation effect is, and there is cache side-channel leakage.

[0119] It should be noted that in actual implementation, for example, the detection processing based on this detection dimension can be used to verify whether the L2 cache bank of the MIG partition under the MIG isolation policy is completely isolated, that is, the probe (e.g., the second tenant) should not observe the cache impact of the signaler (e.g., the first tenant).

[0120] As can be seen, the verification processing based on the detection dimension provided in this embodiment of the present disclosure, by having the second tenant detect the preset memory region and collect the corresponding access latency duration, and then analyzing the data association relationship between it and the preset activity mode in which the first tenant is located, and generating a second verification result characterizing the effectiveness of cache isolation through the obtained first mutual information, can accurately measure the degree of cache information leakage and objectively determine the effectiveness of cache isolation.

[0121] Please refer to Figure 4 This is a third flowchart of the process for obtaining feature data provided in an embodiment of this disclosure. For example... Figure 4 As shown, in some embodiments, data correlation analysis may include analysis from the time-side channel probing dimension; the step S11 of obtaining the operation feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and obtaining the inference feature data of the first tenant, may include the following steps S41-S43.

[0122] Step S41: During the first tenant's execution of the inference task, control the first tenant to execute multiple types of inference requests based on processor hardware resources.

[0123] Step S42: During the process of the first tenant executing multiple types of inference requests, control the second tenant to execute multiple fixed probe requests and obtain the request processing time corresponding to each probe request.

[0124] Step S43: Generate inference feature data based on the time information and request type of each inference request of the first tenant; and generate operation feature data based on the time information and corresponding request processing time of each probe request executed by the second tenant.

[0125] In this embodiment, the step of performing data correlation analysis on the operational feature data and inference feature data to obtain verification results includes: predicting the predicted request type of the inference request executed by the first tenant within a corresponding time period based on the time information of each probe request in the operational feature data and the corresponding request processing duration; determining the matching degree between the obtained predicted request type and the request type of the inference request executed by the first tenant within the corresponding time period based on the time information of each inference request in the inference feature data; and generating a third verification result based on the matching degree; wherein, the third verification result represents the effectiveness of time-side channel isolation under the preset isolation strategy.

[0126] That is, in this embodiment of the disclosure, different types of inference requests can be executed by the first tenant, while a fixed probe request is executed by the second tenant simultaneously, so as to obtain the isolation effectiveness between processor multi-tenants from the probe dimension of the time side channel.

[0127] Specifically, the inference request executed by the first tenant can be at least one of short prompt word requests, long prompt word requests, small-scale batch processing requests, and large-scale batch processing requests. Of course, in actual implementation, it can also include requests between different models. This disclosure does not make any special limitation on the type of inference request executed by the first tenant. For example, in an LLM inference scenario, the inference request can also include prompt word length, number of output tokens, batch size, model type, whether to execute long context inference, and whether to execute high computational request.

[0128] The probe request executed by the second tenant can be of any type, and this disclosure does not impose any special restrictions on it.

[0129] The request processing time can be used to characterize the time elapsed between the second tenant issuing the probe request and receiving the response.

[0130] That is, in this implementation, a third verification result on whether there is time-side channel leakage can be generated by determining whether there is a statistical correlation between the request processing time corresponding to the second tenant's probe request and the inference request executed by the first tenant.

[0131] Specifically, it can be represented as: classification_accuracy = train_classifier(B_latencies → A_request_types).

[0132] Here, A_request_types represents the request type of the inference request executed by the first tenant, that is, information that is not intended to be inferred by the second tenant.

[0133] B_latencies represents the time series of request processing times corresponding to the probe requests executed by the second tenant. For example, it could be B_latencies = [12.1 ms, 12.4 ms, 18.7 ms, 19.2 ms, 13.0 ms]. Typically, the inference requests executed by the first tenant affect shared processors, such as scheduling, queuing, batch merging, caching, and computing resource usage on the GPU, thus causing changes in the request processing times of the probe requests executed by the second tenant. For example, when the first tenant sends an inference request with a short hint, the request processing time for the probe request executed by the second tenant might be 12 ms; when the first tenant sends an inference request with a long hint, the request processing time for the probe request executed by the second tenant might be 20 ms. In this case, the second tenant can infer the request type of the first tenant based on its own request processing time changes.

[0134] `train_classifier(B_latencies → A_request_types)` means training a classifier using the request processing time obtained by the second tenant as input features to predict the inference requests executed by the first tenant within the corresponding time period, thus obtaining the predicted request type. For example, if `B_latencies = [12.1 ms, 12.4 ms, 12.2 ms]`, the classifier can output that the first tenant executed an inference request with a short prompt word; or, if `B_latencies = [19.5 ms, 20.1 ms, 19.8 ms]`, the classifier can output that the first tenant executed an inference request with a long prompt word. In actual implementation, this classifier can be a simple statistical threshold classifier, or it can be a machine learning classifier such as logistic regression, decision tree, random forest, or SVM. This disclosure does not impose any special limitations on it.

[0135] `classification_accuracy` can be used to represent the degree of matching between the predicted request type obtained by the classifier and the actual inference request type of the first tenant. For example, it can be represented as accuracy. If there are a total of 1000 predictions and the classifier predicts correctly 800 times, then `classification_accuracy` can be 80%.

[0136] In some embodiments, when generating a third verification result based on the matching accuracy, the matching accuracy (classification_accuracy) can be compared with the baseline matching accuracy (random_guess) to generate the third verification result. For example, if the first tenant has two request types, its baseline matching accuracy (random_guess) = 1 / 2 = 50%. If there are four request types, its random_guess = 1 / 4 = 25%. If classification_accuracy > random_guess, a third verification result indicating the existence of time-side channel leakage can be generated.

[0137] It should be noted that, in some embodiments, in order to avoid misjudgment, a security threshold can be set, such as 5%, 10% or 15%. Only when classification_accuracy > random_guess + threshold, that is, when the obtained matching degree is much greater than the baseline matching degree, is a third verification result indicating the existence of time-side channel leakage generated.

[0138] As can be seen, based on the method provided in the embodiments of this disclosure, by simultaneously obtaining the request processing time of the second tenant executing a fixed probe request during the process of the first tenant executing different types of inference requests, the correlation between the inference request executed by the first tenant and the change of the request processing time can be predicted, thereby accurately verifying whether there is a time-side channel leakage problem under the prediction isolation strategy.

[0139] Please refer to Figure 5 This is the fourth flowchart for obtaining feature data provided in the embodiments of this disclosure. For example... Figure 5 As shown, in some embodiments, data correlation analysis may include analysis from the cross-tenant resource interference detection dimension; the operation feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, as described in step S11, and the inference feature data of the first tenant may include the following steps S51-S52.

[0140] Step S51: Obtain the baseline inference latency distribution of the second tenant under the condition of exclusively using the processor hardware resources; and obtain the measured inference latency distribution of the second tenant under the condition that the first tenant performs inference tasks and performs different intensity of computational loads.

[0141] Step S52: Generate operational feature data based on the baseline inference delay distribution and the measured inference delay distribution; and obtain the time information of the first tenant's execution of computational loads of various intensities to generate inference feature data.

[0142] In this embodiment, the step of performing data correlation analysis on operational feature data and inference feature data to obtain verification results may include: determining a cross-tenant resource interference index based on the time information in the inference feature data and the baseline inference delay distribution and measured inference delay distribution in the operational feature data; generating a fourth verification result based on the cross-tenant resource interference index; wherein the cross-tenant resource interference index is used to quantify the degree of resource interference between tenants; and the fourth verification result indicates the effectiveness of resource isolation under the preset isolation strategy.

[0143] The baseline inference latency distribution can be used to represent the numerical distribution of inference time when the second tenant occupies all hardware resources alone. For example, it can include any one of the median latency, 90% latency, and 99% latency. Correspondingly, the measured inference latency distribution can be used to represent the numerical distribution of inference latency when the first tenant and the second tenant share processor hardware resources.

[0144] In practice, the cross-tenant resource interference index Interference_Index can be obtained, for example, as follows: Interference_Index = (P99_with_colocated - P99_baseline) / P99_baseline.

[0145] P99_with_colocated can represent 99% of the inference latency in the measured inference latency distribution of the second tenant, that is, the worst latency under the condition of interference from the first tenant; P99_baseline can represent 99% of the inference latency in the baseline inference latency distribution of the second tenant, that is, the worst latency under the condition of monopolizing all the hardware resources of the processor.

[0146] In practice, after obtaining the cross-tenant resource interference index, it can be compared with a preset threshold to generate a fourth verification result indicating the effectiveness of resource isolation, i.e., whether cross-tenant resource interference exists. It should be noted that in practice, two thresholds can be set, for example, 0.1 and 0.5 respectively. If the cross-tenant resource interference index is greater than 0.1 but less than 0.5, it indicates the existence of some cross-tenant resource interference, which can be monitored but isolation remains effective. If the cross-tenant resource interference index is greater than or equal to 0.5, it usually indicates isolation failure, requiring isolation hardening strategies.

[0147] As can be seen, the method provided by the embodiments of this disclosure, by quantifying the degree of resource interference based on the cross-tenant resource interference index, is conducive to unifying the evaluation criteria for resource isolation effect and accurately identifying security defects such as resource crowding.

[0148] Please refer to Figure 6 This is the fifth flowchart for obtaining feature data provided in the embodiments of this disclosure. For example... Figure 6 As shown, in some embodiments, data correlation analysis may include analysis from the data interconnect side-channel probing dimension; processor hardware resources may include data interconnect resources, which may be, for example, inter-card data interconnect resources between multiple GPU nodes, such as at least one of NVLink, MTLink, and PCIe. In this embodiment, the processor may be configured with a first isolation region and a second isolation region that are isolated from each other. Step S11, which involves obtaining the operation feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and obtaining the inference feature data of the first tenant, may include the following steps S61-S62.

[0149] Step S61: During the first tenant's inference task, the first tenant is controlled to perform cross-processor data transmission processing based on data interconnection resources in the first isolation zone, and inference feature data is generated according to the time information corresponding to the cross-processor data transmission processing.

[0150] Step S62: During the cross-processor data transfer process performed by the first tenant, the second tenant is controlled to perform multiple probe processes for data interconnect resources in the second isolation zone and obtain the probe duration. Based on the obtained multiple probe durations, operation feature data is generated.

[0151] In this embodiment, the step of performing data correlation analysis on the operational feature data and inference feature data to obtain a verification result may include: determining a second mutual information between the operational feature data and the inference feature data based on the time information in the inference feature data and multiple detection durations in the operational feature data, and generating a fifth verification result based on the second mutual information; wherein the fifth verification result represents the effectiveness of interconnection resource isolation under the preset isolation strategy.

[0152] In the process of implementing this disclosure, the applicant discovered that data interconnect resources in a multi-GPU environment may also become a path for information leakage across virtual machines or across tenants. Therefore, it is necessary to verify the isolation effectiveness in a multi-tenant inference environment from the perspective of probing the data interconnect side channel.

[0153] In this implementation, while the first tenant is performing cross-processor data transmission based on data interconnect resources in its first isolation zone, the second tenant can be simultaneously controlled to perform multiple probe processes targeting data interconnect resources in the second isolation zone. The inference feature data of the first tenant is generated based on the time information of the data transmission process performed by the first tenant, and the operation feature data is generated based on the probe duration obtained by the second tenant.

[0154] For example, when the first tenant performs cross-GPU data transfer through data interconnect resources such as NVLink, MTLink, or PCIe, the second tenant can be controlled to perform fixed NVLink, MTLink, or PCIe probe operations (i.e., perform a set of cross-GPU transfers of known size) and obtain their corresponding probe durations in another isolation zone on the same physical node.

[0155] Subsequently, it can be determined whether a data interconnect side channel exists by judging whether the probing time of the second tenant is significantly longer than that of the first tenant when the first tenant is idle during the large-scale cross-GPU transfer.

[0156] The relevant processing for obtaining the second mutual information and generating the fifth verification result based on the second mutual information is explained in the section on the first mutual information above, and will not be repeated here.

[0157] As can be seen, based on the method provided in this disclosure, the problem of hidden side-channel leakage across virtual machines and across tenants in shared interconnect links such as NVLink, MTLink, and PCIe under multi-GPU distributed inference architecture is addressed. By asynchronously collecting transmission timing data and interconnect probe duration sequences in mutually isolated dual regions, and reusing mature mutual information calculation methods to quantify the coupling correlation between transmission behavior and probe latency, the isolation and protection effect at the interconnect channel level can be accurately determined. Detection based on this probe dimension aligns with the real-world business operation mechanism of multi-GPU cluster distributed training and tensor parallel inference, thereby accurately identifying hidden information theft channels formed by interconnect link bandwidth contention and transmission timing disturbances.

[0158] It should be noted that the above describes five detection dimensions—memory remnant detection, cache side-channel detection, time side-channel detection, cross-tenant resource interference detection, and data interconnect side-channel detection—under a preset isolation strategy in a multi-tenant processor inference environment, such as the MIG partition isolation strategy. Corresponding verification results (first, second, third, fourth, and fifth verification results) are generated. In practice, one or more of these five detection dimensions can be used simultaneously to verify the isolation security of the multi-tenant inference environment to prevent cross-tenant data leakage. Alternatively, all five detection dimensions can be used simultaneously. This approach, because it includes detection of any processor hardware resource that multiple tenants may share, can not only detect isolation flaws with known attack knowledge but also promptly identify isolation flaws with various unknown attack methods, thus preventing cross-tenant data leakage.

[0159] It should be noted that, in some embodiments, the verification result obtained in step S12 may include verification scores for at least one of the following detection dimensions: memory residue detection dimension, cache side-channel detection dimension, time side-channel detection dimension, cross-tenant resource interference detection dimension, and data interconnection side-channel detection dimension; in this embodiment, the method may further include: performing weighted summation on each verification score in the verification result according to the weights corresponding to different detection dimensions to obtain a target isolation security score; wherein, the target isolation security score is used to represent the degree of isolation effectiveness between multiple tenants under the preset isolation strategy.

[0160] For example, let Memory_Score represent the verification score in the first verification result used to indicate the effectiveness of memory isolation, which can be determined, for example, based on the ratio between the second data volume and the first data volume; let Cache_Score represent the verification score in the second verification result used to indicate the effectiveness of cache isolation, which can be determined, for example, based on the first mutual information; let Timing_Score represent the verification score in the third verification result used to indicate the degree of time-side channel leakage, which can be determined, for example, based on the matching degree between the predicted request type and the request type of the inference request executed by the first tenant within the corresponding time period; let Interference_Score represent the verification score in the fourth verification result used to indicate the degree of resource interference; and let Interconnect_Score represent the verification score in the fifth verification result used to indicate the degree of data interconnect resource leakage, with w1, w2, w3, w4, and w5 representing the weights corresponding to different detection dimensions, respectively; then the target isolation security score ISS can be obtained, for example, by the following formula:

[0161] ISS=w1×Memory_Score+w2×Cache_Score+w3×Timing_Score+w4×Interference_Score+w5×Interconnect_Score.

[0162] Wherein, Memory_Score∈[0,1], 0 indicates complete leakage, 1 indicates complete isolation; Cache_Score∈[0,1]: 0 indicates complete information leakage, 1 indicates no detectable side channel; Timing_Score∈[0,1]: 0 indicates accurate classification, 1 indicates no timing side channel; Interference_Score∈[0,1]: 0 indicates complete interference, 1 indicates no performance impact; Interconnect_Score∈[0,1]: 0 indicates complete eavesdropping of the interconnect, 1 indicates no interconnect side channel; The values ​​of w1, w2, w3, w4 and w5 can be set as needed, and this disclosure does not impose any special restrictions on them. For example, in a multi-GPU inference scenario, the weight w5 of the detection dimension of the data interconnect side channel can be appropriately increased, while in a single-GPU MIG application scenario, w5 can be reduced to 0.10.

[0163] It should be noted that, in some embodiments, when the target isolation security score (ISS) is ≥ 0.9, a rating result predicting the isolation strategy has a first isolation level (i.e., strong isolation effect) can be generated, which is suitable for high security requirements scenarios; when ISS is ≤ 0.7 and < 0.9, a rating result predicting the isolation strategy has a second isolation level (i.e., good isolation effect) can be generated, which is suitable for general multi-tenant scenarios; when ISS is ≤ 0.5 and < 0.7, a rating result predicting the isolation strategy has a third isolation level (i.e., weak isolation effect) can be generated, indicating that the currently used preset isolation strategy has an exploitable leakage path; and when ISS is < 0.5, a rating result predicting the isolation strategy has a fourth isolation level (i.e., isolation failure) can be generated, which is a rating result that is not recommended for multi-tenant deployment.

[0164] In practice, after obtaining the corresponding isolation level based on the target isolation security score, the isolation level can be output to facilitate user decision-making and avoid leakage of multi-tenant privacy data.

[0165] In addition, in some embodiments, the method may further include: determining a corresponding differentiated isolation hardening strategy based on the verification score corresponding to each detection dimension; updating the preset isolation strategy based on the differentiated isolation hardening strategy; and performing isolation processing between different tenants in the processor based on the updated preset isolation strategy.

[0166] In this implementation, the step of determining the corresponding differentiated isolation and hardening strategy based on the verification score corresponding to each detection dimension includes at least one of the following A1-A5.

[0167] A1, if the verification score corresponding to the residual memory detection dimension is less than or equal to the first preset security threshold, a first isolation hardening strategy is generated; the first isolation hardening strategy is used to instruct the processor to trigger the execution of a zeroing operation for the memory resources after detecting the release of memory resources of any tenant.

[0168] The first preset safety threshold can be, for example, 0.5, or it can be set as needed; this disclosure does not impose any special limitations on it.

[0169] Specifically, if the verification score corresponding to the memory remnant detection dimension is less than or equal to the first preset security threshold, such as 0.5, and a memory remnant problem is confirmed, then, for example, after the business nodes such as releasing KVCache memory pages, MIG partition memory reallocation, and CUDA inference context destruction are completed in inference frameworks such as vLLM and / or TGI, a batch memory clearing operation can be automatically triggered, and the reclaimed memory space can be initialized and overwritten. For example, the batch memory clearing operation can be triggered by executing the standard CUDA API: cudaMemset after free. At the same time, it can be combined with the instruction: CUDA_LAUNCH_BLOCKING with memset, and the environment variable CUDA_LAUNCH_BLOCKING=1 can be configured to block and wait for the cudaMemset memory clearing kernel to complete its execution before allowing subsequent memory allocation operations to completely eliminate the residual feature data of the former tenant and block the memory remnant leakage path from the hardware memory release link.

[0170] A2, if the verification score corresponding to the cache side channel detection dimension is less than or equal to the second preset security threshold, generate a second isolation hardening strategy; the second isolation hardening strategy is used to instruct the processor to perform at least one of the following operations: enable multi-instance hardware partition isolation operation, perform cache refresh operation between batches, and reduce batch merging granularity operation.

[0171] The second preset safety threshold can be, for example, 0.5, or it can be set as needed; this disclosure does not impose any special limitations on it.

[0172] Specifically, if the verification score of the cache side-channel detection dimension is less than or equal to the second preset security threshold, such as 0.5, and it is determined that there is a cache contention side-channel risk, then a second isolation hardening strategy can be generated.

[0173] The second isolation and hardening strategy can be, for example, enabling MIG hardware partitioning isolation so that different tenants can exclusively enjoy independent L2Cache cache resources and avoid cross-tenant cache line preemption; or, actively performing a global cache refresh operation between multiple rounds of batch processing inference to clear the cache traces of the current round of computation; or, reducing the granularity of framework batch processing merging to avoid large batches of computations crowding out cache resources, weakening the cache timing observability characteristics, and resisting cache side-channel theft attacks.

[0174] A3, if the verification score corresponding to the time-side channel detection dimension is less than or equal to the third preset security threshold, a third isolation hardening strategy is generated; the third isolation hardening strategy is used to instruct the processor to perform time padding processing that increases the preset inference duration for any tenant's inference request.

[0175] The third preset safety threshold can be, for example, 0.5, or it can be set as needed; this disclosure does not impose any special limitations on it.

[0176] The preset inference duration can be set as needed. For example, for short text inference requests, the preset inference duration can be 8ms; for long document or long context inference requests, the preset inference duration can be, for example, 30ms.

[0177] Specifically, if the verification score of the time-side channel detection dimension is less than or equal to the third preset security threshold, such as 0.5, it can be determined that there is a risk that the timing information can be inferred, and then a third isolation hardening strategy can be generated.

[0178] The third isolation and hardening strategy can be, for example, applying constant-time padding to inference requests of varying lengths and batch sizes, thereby filling in all inference tasks to a uniform maximum time consumption by padding the computation cycle, smoothing out the latency differences between inference tasks of different tenants, and eliminating the timing side channel that can be used for reverse inference of service characteristics.

[0179] A4, if the verification score corresponding to the cross-tenant resource interference detection dimension is less than or equal to the fourth preset security threshold, generate the fourth isolation hardening strategy; the fourth isolation hardening strategy is used to instruct the processor to execute at least one of the following: adjust the multi-instance hardware partition space capacity, limit the number of concurrent tenants, and enable the computing power service quality control QoS.

[0180] The fourth preset safety threshold can be, for example, 0.5, or it can be set as needed; this disclosure does not impose any special limitations on it.

[0181] Specifically, if the verification score of the cross-tenant resource interference detection dimension is less than or equal to the fourth preset security threshold, such as 0.5, it can be determined that there is a problem of computing power and bandwidth resource squeezing, and then the fourth isolation and hardening strategy can be generated.

[0182] The fourth isolation and hardening strategy can be: dynamically adjusting the memory capacity and computing power ratio of the MIG hardware partition to balance the tenant resource quota; or, limiting the maximum number of concurrent tenants within a single physical GPU to avoid tenant overload; or, enabling GPU computing power QoS bandwidth limiting and computing power weight scheduling to ensure the basic inference performance of low-load tenants and suppress resource interference jitter caused by high-load tenants.

[0183] A5, if the verification score corresponding to the data interconnect side channel detection dimension is less than or equal to the fifth preset security threshold, a fifth isolation hardening strategy is generated; the fifth isolation hardening strategy is used to instruct the processor to execute at least one of the following: binding dedicated data interconnect resources, enabling interconnect layer traffic filling, and restricting cross-processor communication scheduling alignment.

[0184] The fifth preset safety threshold can be, for example, 0.5, or it can be set as needed; this disclosure does not impose any special limitations on it.

[0185] Specifically, if the verification score of the data interconnection side channel detection dimension is less than or equal to the fifth preset security threshold, such as 0.5, it can be determined that there is a risk of side channel leakage in the multi-card interconnection channel, thereby generating the fifth isolation and hardening strategy.

[0186] The fifth isolation hardening strategy can be: binding dedicated data interconnect resources. For example, dedicated NVLink channels, dedicated PCIe link ports, and dedicated bandwidth quotas can be statically bound to different isolated tenants and different isolated virtual machines. By adopting a dedicated link scheduling method, cross-tenant sharing of interconnect channel bandwidth is prohibited, thereby avoiding the leakage of timing characteristics due to bandwidth contention.

[0187] Alternatively, the fifth isolation hardening strategy can be to enable interconnect layer traffic padding. For example, during idle time slots when a tenant does not have cross-card transmission tasks, a constant-rate padding of fixed length can be injected into the tenant's dedicated interconnect channel to mask the traffic fluctuations and timing characteristics of the actual business transmission, thereby concealing cross-card communication behavior.

[0188] Alternatively, this fifth isolation hardening strategy could be: restricting cross-processor communication scheduling alignment. For example, time slot alignment scheduling and / or batch unified packet sending scheduling could be implemented for multi-tenant cross-GPU communication requests, thereby disrupting the random transmission timing differences observable by attackers and masking the interconnection side-channel characteristics caused by cross-card communication scheduling disorder.

[0189] As can be seen, through the above-mentioned dimensional and differentiated precise reinforcement methods, the embodiments of this disclosure can specifically strengthen the isolation defects of different hardware levels and different leakage mechanisms, no longer using a uniform and coarse isolation configuration, and can effectively repair various security risks such as video memory residue, cache side channel, timing leakage, resource interference, and interconnect channel side channel, and continuously iterate and optimize the isolation strategy of multi-tenant processors, thereby significantly improving the overall isolation security and resistance to side channel attacks in multi-tenant GPU inference scenarios.

[0190] After generating the corresponding differentiated isolation and hardening strategy, the preset isolation strategy can be updated to avoid privacy data leakage issues between tenants and improve the isolation security of the multi-tenant processor inference environment.

[0191] It should be noted that, in some embodiments, the method may further include: triggering the execution of isolation security verification processing for the multi-tenant processor upon detecting that the processor's driver or firmware update is complete.

[0192] In practice, the above-mentioned multi-dimensional detection and processing can be continuously verified and integrated for deployment. After the processor driver, firmware or framework is updated, automated verification can be performed based on the multi-dimensional detection and processing. Security verification reports can be generated according to possible leakage paths, assessment levels and corresponding differentiated isolation and hardening strategies for users to view.

[0193] In some embodiments, the method may further include: employing an asynchronous parallel strategy to perform data correlation analysis from at least two of the following detection dimensions: video memory residual detection dimension, cache side-channel detection dimension, time side-channel detection dimension, cross-tenant resource interference detection dimension, and data interconnect side-channel detection dimension.

[0194] Specifically, in this embodiment, an independent detection engine can be configured to perform multi-dimensional detection processing. For example, data correlation analysis can be performed simultaneously based on the above five dimensions. Within the same physical hardware accelerator or shared interconnect domain, the detection engine forces the construction of asynchronous concurrent execution timing between tenants, enabling the probe (corresponding to the second tenant in this embodiment) to accurately sample the microarchitectural state retention or delay disturbance generated by the signal (corresponding to the first tenant in this embodiment) on shared hardware resources (L2 Cache or interconnect bandwidth). This anchors the verification logic of this embodiment to the real-time acquisition and quantization sampling level of the processor hardware microarchitectural physical state, distinguishing it from the pure software-level runtime log parsing and upper-layer business logic verification methods in related technologies. Based on this embodiment, the isolation effectiveness can be detected from the hardware layer, thereby improving the accuracy of the detection results.

[0195] For example, independent acquisition threads and analysis subprocesses are created for the following dimensions: memory remnant detection, cache-side channel detection, time-side channel detection, cross-tenant resource interference detection, and data interconnect-side channel detection. All detection units are triggered synchronously by a unified global clock, and each independently completes feature data acquisition, raw data caching, and correlation calculation for its corresponding dimension. The calculation tasks for each dimension are executed in parallel based on CPU multi-core computing power and idle GPU computing resources. After the calculation of each dimension is completed, the corresponding sub-item verification scores are summarized to calculate the overall isolation security score. Each detection process runs independently without interference. For example, memory feature code read / write operations, cache line latency detection sampling, inference latency acquisition, baseline latency statistics, and NVLink / PCIe link detection can be carried out simultaneously without waiting for the previous dimension to be completed before executing the next dimension's detection process. Even if sampling retry or data calculation time fluctuations occur in a single dimension detection, it will not hinder the normal detection operations of other dimensions.

[0196] As can be seen, the method provided by this disclosure, compared with detection methods that rely on only a single detection dimension or execute multiple detection dimensions serially, performs asynchronous parallel detection based on multiple detection dimensions. This fully utilizes the multi-core computing resources of the server, significantly shortening the overall time consumption of multi-dimensional isolation security detection. Especially after processor firmware or driver updates, this verification method can automatically and quickly perform regression verification processing, thereby improving the execution efficiency of security verification. In addition, by completing sampling detection based on the native perturbation signals of the hardware microarchitecture, the detection data all come from the actual operating state of the hardware. Compared with software log verification methods, the method provided by this disclosure is not easily tampered with or blocked by tenant-side programs, making the authenticity and feasibility of the verification results higher. Furthermore, the feature acquisition and analysis logic of each detection dimension are run in parallel through asynchronous physical isolation, which can avoid the problem of the overall detection process being stuck and data distortion caused by single-dimensional anomalies, thereby improving the stability of the verification method.

[0197] Refer to Table 1 below, which is a schematic table based on internal security testing experiments to illustrate the effects of this disclosure.

[0198] Table 1

[0199]

[0200] As can be seen, the method provided by the embodiments of this disclosure can intuitively and accurately verify the isolation security in a multi-tenant processor from at least one detection dimension, and can provide a security verification report that can be analyzed. It can also provide a differentiated hardening strategy, so that users can update the currently used isolation strategy based on the differentiated hardening strategy to improve the isolation security between multi-tenants in the multi-tenant processor inference environment.

[0201] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0202] In addition, this disclosure also provides an isolation security verification device, electronic device, and computer-readable storage medium for multi-tenant processors. All of the above can be used to implement any of the isolation security verification methods for multi-tenant processors provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding descriptions in the method section and will not be repeated here.

[0203] Figure 7 This is a block diagram of an isolated security verification device for a multi-tenant processor provided in an embodiment of the present disclosure.

[0204] Reference Figure 7 This disclosure provides an isolation security verification device for a multi-tenant processor, which includes an acquisition module 71 and a verification module 72.

[0205] The acquisition module 71 is used to acquire, in response to the first tenant performing an inference task, operation feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and inference feature data of the first tenant.

[0206] Wherein, the first tenant and the second tenant are different tenants isolated in the processor based on a preset isolation strategy; the operation feature data is used to represent the operation behavior; the inference feature data is used to represent the activity status of the first tenant in the inference task;

[0207] The verification module 72 is used to perform data correlation analysis on the operational feature data and the inference feature data, and generate verification results. The data correlation analysis is used to analyze the data change trends and / or numerical feature correspondence between the operational feature data and the inference feature data. The verification results indicate the isolation effectiveness between multiple tenants under the preset isolation strategy.

[0208] In some embodiments, when performing data correlation analysis, the verification module 72 can analyze from at least one of the following detection dimensions: memory residue detection dimension, used to detect the risk of cross-tenant data residue leakage caused by memory resources not being cleared after release; cache side-channel detection dimension, used to detect the risk of side-channel attacks that can steal tenant computing characteristics caused by tenant contention for processor cache resources; time side-channel detection dimension, used to detect the risk of stealing time-series information that infers tenant service types based on differences in inference task processing time; cross-tenant resource interference detection dimension, used to detect the risk of resource crowding out caused by tenants preempting hardware computing resources, resulting in a decrease in the service performance of other tenants; and data interconnect side-channel detection dimension, used to detect the risk of leakage of cross-virtual machine and / or cross-tenant communication information caused by contention for multi-processor interconnect channels.

[0209] In some embodiments, the processor hardware resources include video memory resources; the data correlation analysis includes analysis from the perspective of video memory remnant detection; when the acquisition module 71 acquires the operation feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and acquires the inference feature data of the first tenant, it can be used to: during the first tenant's execution of an inference task, control the first tenant to write target feature pattern data to the target video memory region of the allocated video memory resources, and generate the inference feature data according to the target feature pattern data; after the first tenant completes the inference task and releases the allocated video memory resources, control the second tenant to perform a data reading operation on the target video memory region without performing a video memory initialization operation, and generate the operation feature data according to the data reading result. In this embodiment, when the verification module 72 performs data correlation analysis on the operation feature data and the inference feature data to obtain the verification result, it can be used to: generate a first verification result by determining whether the operation feature data includes data that matches the target feature pattern data in the inference feature data; wherein, the first verification result indicates the effectiveness of video memory isolation under the preset isolation strategy.

[0210] In some embodiments, the inference feature data further includes a first data volume, which represents the amount of target feature pattern data written by the first tenant to the target memory region; the operation feature data further includes a second data volume, which represents the amount of target feature pattern data read by the second tenant from the target memory region; when the verification module 72 generates a first verification result by determining whether the operation feature data includes data that matches the target feature pattern data in the inference feature data, it can be used to: determine the ratio between the second data volume and the first data volume when it is determined that the operation feature data includes the target feature pattern data, and generate the first verification result based on the ratio, wherein the ratio is used to quantify the degree of residual memory leakage.

[0211] In some embodiments, the processor hardware resources further include processor cache resources; the data correlation analysis includes analysis from the cache-side channel probing dimension; when the acquisition module 71 acquires the operation feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and acquires the inference feature data of the first tenant, it can be used to: control the first tenant to perform processor operation processing based on the processor cache resources during the first tenant's inference task; control the second tenant to perform multiple probing processes on a preset memory region in the processor cache and acquire the access latency duration of each probing process during the first tenant's execution of the processor operation processing; generate the inference feature data according to the time information of the first tenant's execution of the processor operation processing; and generate the operation feature data according to the multiple access latency durations obtained. In this embodiment, when the verification module 72 performs data correlation analysis on the operation feature data and the inference feature data to obtain the verification result, it can be used to: determine the correspondence between each access delay duration among the plurality of access delay durations and the preset activity mode based on the time information in the inference feature data; determine the first mutual information between the operation feature data and the inference feature data based on the correspondence; and generate a second verification result based on the first mutual information; wherein the second verification result indicates the effectiveness of cache isolation under the preset isolation strategy.

[0212] In some embodiments, the data correlation analysis includes analysis from the time-side channel probing dimension; when the acquisition module 71 acquires the operation feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and acquires the inference feature data of the first tenant, it can be used to: control the first tenant to execute multiple types of inference requests based on the processor hardware resources during the first tenant's inference task; control the second tenant to execute multiple fixed probing requests during the first tenant's execution of the multiple types of inference requests, and acquire the request processing time corresponding to each probing request; generate the inference feature data according to the time information of each inference request of the first tenant and the request type of each inference request; and generate the operation feature data according to the time information of each probing request executed by the second tenant and the corresponding request processing time. In this embodiment, when the verification module 72 performs data correlation analysis on the operation feature data and the inference feature data to obtain the verification result, it can be used to: predict the predicted request type of the inference request executed by the first tenant within a corresponding time period based on the time information of each probe request in the operation feature data and the corresponding request processing duration; determine the matching degree between the obtained predicted request type and the request type of the inference request executed by the first tenant within the corresponding time period based on the time information of each inference request in the inference feature data; and generate a third verification result based on the matching degree; wherein the third verification result represents the effectiveness of time-side channel isolation under the preset isolation strategy.

[0213] In some embodiments, the data correlation analysis includes analysis from the cross-tenant resource interference detection dimension; when the acquisition module 71 acquires the operation feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and acquires the inference feature data of the first tenant, it can be used to: acquire the baseline inference latency distribution of the second tenant when it exclusively occupies the processor hardware resources; acquire the measured inference latency distribution of the second tenant when the first tenant performs inference tasks and performs computational loads of different intensities; generate the operation feature data based on the baseline inference latency distribution and the measured inference latency distribution; and acquire the time information of the first tenant performing computational loads of various intensities to generate the inference feature data. In this embodiment, when the verification module 72 performs data correlation analysis on the operational feature data and the inference feature data to obtain the verification result, it can be used to: determine the cross-tenant resource interference index based on the time information in the inference feature data and the baseline inference delay distribution and the measured inference delay distribution in the operational feature data; generate a fourth verification result based on the cross-tenant resource interference index; wherein, the cross-tenant resource interference index is used to quantify the degree of resource interference between tenants; the fourth verification result indicates the effectiveness of resource isolation under the preset isolation strategy.

[0214] In some embodiments, the processor hardware resources include data interconnect resources; the processor is configured with a first isolation zone and a second isolation zone that are isolated from each other; the data correlation analysis includes analysis from the data interconnect side channel probing dimension; when the acquisition module 71 acquires the operation feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and acquires the inference feature data of the first tenant, it can be used to: control the first tenant to perform cross-processor data transmission processing in the first isolation zone based on the data interconnect resources during the first tenant's inference task, and generate the inference feature data according to the time information corresponding to the cross-processor data transmission processing; control the second tenant to perform multiple probing processes for the data interconnect resources in the second isolation zone during the first tenant's cross-processor data transmission processing, and acquire the probing duration, and generate the operation feature data according to the multiple acquired probing durations. In this embodiment, when the verification module 72 performs data correlation analysis on the operation feature data and the inference feature data to obtain the verification result, it can be used to: determine the second mutual information between the operation feature data and the inference feature data based on the time information in the inference feature data and the multiple detection durations in the operation feature data, and generate a fifth verification result based on the second mutual information; wherein, the fifth verification result indicates the effectiveness of interconnection resource isolation under the preset isolation strategy.

[0215] In some embodiments, the verification result includes verification scores for at least two of the following detection dimensions: memory residue detection dimension, cache side-channel detection dimension, time side-channel detection dimension, cross-tenant resource interference detection dimension, and data interconnect side-channel detection dimension. The device also includes a scoring module, which can be used to: perform weighted summation on each verification score in the verification result according to the weights corresponding to different detection dimensions to obtain a target isolation security score. The target isolation security score is used to represent the degree of isolation effectiveness between multiple tenants under the preset isolation strategy.

[0216] In some embodiments, the device may further include a policy update module, which may be used to: determine a corresponding differentiated isolation hardening policy based on the verification score corresponding to each detection dimension; update the preset isolation policy based on the differentiated isolation hardening policy; and perform isolation processing between different tenants in the processor based on the updated preset isolation policy.

[0217] In some embodiments, when the policy update module determines the corresponding differentiated isolation and hardening strategy based on the verification score corresponding to each detection dimension, it can be used to: generate a first isolation and hardening strategy when the verification score corresponding to the memory residue detection dimension is less than or equal to a first preset security threshold; the first isolation and hardening strategy is used to instruct the processor to trigger a zeroing operation for the memory resources after detecting the release of memory resources of any tenant; generate a second isolation and hardening strategy when the verification score corresponding to the cache side channel detection dimension is less than or equal to a second preset security threshold; the second isolation and hardening strategy is used to instruct the processor to perform at least one of: enabling multi-instance hardware partition isolation operation, performing cache refresh operation between batches, and reducing batch merging granularity operation; when the verification score corresponding to the time side channel detection dimension is less than or equal to a third preset security threshold, generate a second isolation and hardening strategy; the second isolation and hardening strategy is used to instruct the processor to perform at least one of: enabling multi-instance hardware partition isolation operation, performing cache refresh operation between batches, and reducing batch merging granularity operation; when the verification score corresponding to the time side channel detection dimension is less than or equal to a third preset security threshold, generate a second isolation and hardening strategy. Given a security threshold, a third isolation hardening strategy is generated. This third isolation hardening strategy instructs the processor to perform time padding processing that increases the inference duration for any tenant's inference request. If the verification score corresponding to the cross-tenant resource interference detection dimension is less than or equal to a fourth preset security threshold, a fourth isolation hardening strategy is generated. This fourth isolation hardening strategy instructs the processor to perform at least one of the following: adjusting the multi-instance hardware partition space capacity, limiting the number of concurrent tenants, and enabling QoS (Quality of Service) control for computing power. If the verification score corresponding to the data interconnect side channel detection dimension is less than or equal to a fifth preset security threshold, a fifth isolation hardening strategy is generated. This fifth isolation hardening strategy instructs the processor to perform at least one of the following: binding dedicated data interconnect resources, enabling interconnect layer traffic padding, and limiting cross-processor communication scheduling alignment.

[0218] In some embodiments, the apparatus further includes an update triggering module, which can be used to trigger the execution of isolation security verification processing for multi-tenant processors when a driver or firmware update of the processor is detected to be complete.

[0219] In some embodiments, the apparatus further includes an asynchronous detection module, which can be used to perform the data correlation analysis from at least two of the following detection dimensions: the video memory residual detection dimension, the cache side-channel detection dimension, the time side-channel detection dimension, the cross-tenant resource interference detection dimension, and the data interconnect side-channel detection dimension, using an asynchronous parallel strategy.

[0220] Figure 8 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.

[0221] Reference Figure 8 This disclosure provides an electronic device, which includes: at least one processor 701; at least one memory 702; and one or more I / O interfaces 703 connected between the processor 701 and the memory 702; wherein the memory 702 stores one or more computer programs that can be executed by at least one processor 701, and the one or more computer programs are executed by at least one processor 701 to enable at least one processor 701 to execute the above-described multi-tenant processor isolation security verification method.

[0222] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the aforementioned multi-tenant processor isolation security verification method. The computer-readable storage medium may be volatile or non-volatile.

[0223] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described multi-tenant processor isolation security verification method.

[0224] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0225] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0226] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0227] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0228] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0229] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0230] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0231] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0232] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0233] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.

Claims

1. A method for isolating and verifying security in a multi-tenant processor, characterized in that, include: In response to the first tenant performing an inference task, the system obtains operation feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and obtains inference feature data of the first tenant. Wherein, the first tenant and the second tenant are different tenants isolated in the processor based on a preset isolation strategy; the operation feature data is used to represent the operation behavior; the inference feature data is used to represent the activity status of the first tenant in the inference task; A data correlation analysis is performed on the operational feature data and the inference feature data to generate a verification result. The data correlation analysis is used to analyze the data change trends and / or numerical feature correspondences between the operational feature data and the inference feature data. The verification result indicates the isolation effectiveness between multiple tenants under the preset isolation strategy.

2. The method according to claim 1, characterized in that, The data correlation analysis includes analysis from at least one of the following detection dimensions: The memory residue detection dimension is used to detect the risk of cross-tenant data leakage caused by memory residue not being cleared after memory resources are released; The cache side-channel probing dimension is used to detect the risk of side-channel attacks that could steal tenant computing features caused by contention for processor cache resources between tenants; The time-side channel detection dimension is used to detect the risk of stealing time-series information that infers the tenant's service type based on differences in inference task processing time; Cross-tenant resource interference detection dimension is used to detect the risk of resource crowding out caused by tenants preempting hardware computing resources, resulting in a decline in the business performance of other tenants; The data interconnect side channel detection dimension is used to detect the risk of leakage of cross-virtual machine and / or cross-tenant communication information caused by multiprocessor interconnect channel contention.

3. The method according to claim 2, characterized in that, The processor hardware resources include video memory resources; the data correlation analysis includes analysis from the perspective of video memory residual detection. The step of obtaining operational feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and obtaining inference feature data of the first tenant, includes: During the execution of the inference task by the first tenant, the first tenant is controlled to write target feature pattern data into the target memory region of the allocated video memory resources, and the inference feature data is generated based on the target feature pattern data. After the first tenant completes its inference task and releases the allocated video memory resources, the second tenant is controlled to perform a data reading operation on the target video memory region without performing a video memory initialization operation, and the operation feature data is generated based on the data reading result. The step of performing data correlation analysis on the operational feature data and the inference feature data to obtain the verification result includes: A first verification result is generated by determining whether the operational feature data includes data that matches the target feature pattern data in the inference feature data; wherein the first verification result indicates the effectiveness of memory isolation under the preset isolation strategy.

4. The method according to claim 3, characterized in that, The inference feature data also includes a first data volume, which represents the amount of target feature pattern data written by the first tenant to the target memory region; the operation feature data also includes a second data volume, which represents the amount of target feature pattern data read by the second tenant from the target memory region. The step of generating a first verification result by determining whether the operational feature data includes data that matches the target feature pattern data in the inference feature data includes: If the target feature pattern data is determined to be included in the operational feature data, the ratio between the second data volume and the first data volume is determined, and the first verification result is generated based on the ratio, wherein the ratio is used to quantify the degree of residual leakage of video memory.

5. The method according to claim 2, characterized in that, The processor hardware resources also include processor cache resources; the data correlation analysis includes analysis from the cache-side channel probing dimension; The step of obtaining operational feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and obtaining inference feature data of the first tenant, includes: During the execution of the inference task by the first tenant, the first tenant is controlled to perform processor operations based on the processor cache resources in a preset activity mode; During the first tenant's execution of the processor's computational processing, the second tenant is controlled to perform multiple probe processes on a preset memory region in the processor cache and obtain the access latency duration of each probe process. The inference feature data is generated based on the time information of the first tenant performing the processor operation; and the operation feature data is generated based on the obtained multiple access delay durations. The step of performing data correlation analysis on the operational feature data and the inference feature data to obtain the verification result includes: Based on the time information in the inference feature data, determine the correspondence between each of the multiple access delay durations and the preset activity mode; Based on the correspondence, a first mutual information between the operation feature data and the inference feature data is determined, and a second verification result is generated based on the first mutual information; wherein, the second verification result represents the cache isolation effectiveness under the preset isolation strategy.

6. The method according to claim 2, characterized in that, The data correlation analysis includes analysis from the time-side channel detection dimension; The step of obtaining operational feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and obtaining inference feature data of the first tenant, includes: During the execution of the inference task by the first tenant, the first tenant is controlled to execute various types of inference requests based on processor hardware resources; During the process of the first tenant executing the various types of inference requests, the second tenant is controlled to execute multiple fixed probe requests, and the request processing time corresponding to each probe request is obtained; The inference feature data is generated based on the time information and request type of each inference request of the first tenant; and the operation feature data is generated based on the time information of each probe request executed by the second tenant and the corresponding request processing time. The step of performing data correlation analysis on the operational feature data and the inference feature data to obtain the verification result includes: Based on the time information of each probe request in the operation feature data and the corresponding request processing time, predict the predicted request type of the inference request executed by the first tenant within the corresponding time period. Based on the time information of each inference request in the inference feature data, determine the degree of matching between the obtained predicted request type and the request type of the inference request executed by the first tenant within the corresponding time period; Based on the matching degree, a third verification result is generated; wherein, the third verification result represents the effectiveness of time-side channel isolation under the preset isolation strategy.

7. The method according to claim 2, characterized in that, The data correlation analysis includes analysis from the cross-tenant resource interference detection dimension; The step of obtaining operational feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and obtaining inference feature data of the first tenant, includes: Obtain the baseline inference latency distribution of the second tenant when it exclusively uses the processor hardware resources of the processor; and obtain the measured inference latency distribution of the second tenant when the first tenant performs inference tasks and performs different intensity of computational loads. Based on the baseline inference latency distribution and the measured inference latency distribution, the operational feature data is generated; and the time information of the first tenant executing computational loads of various intensities is obtained to generate the inference feature data. The step of performing data correlation analysis on the operational feature data and the inference feature data to obtain the verification result includes: Based on the time information in the inference feature data and the baseline inference delay distribution and the measured inference delay distribution in the operation feature data, the cross-tenant resource interference index is determined. A fourth verification result is generated based on the cross-tenant resource interference index. The cross-tenant resource interference index is used to quantify the degree of resource interference between tenants; the fourth verification result indicates the effectiveness of resource isolation under the preset isolation strategy.

8. The method according to claim 2, characterized in that, The processor hardware resources include data interconnect resources; the processor is configured with a first isolation zone and a second isolation zone that are mutually isolated; the data correlation analysis includes analysis from the perspective of side channel detection of the data interconnect. The step of obtaining operational feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and obtaining inference feature data of the first tenant, includes: During the execution of the inference task by the first tenant, the first tenant is controlled to perform cross-processor data transmission processing in the first isolation zone based on the data interconnection resources, and the inference feature data is generated according to the time information corresponding to the cross-processor data transmission processing. During the cross-processor data transfer process performed by the first tenant, the second tenant is controlled to perform multiple probe processes for the data interconnect resource in the second isolation area, and the probe duration is obtained. Based on the multiple probe durations obtained, the operation feature data is generated. The step of performing data correlation analysis on the operational feature data and the inference feature data to obtain the verification result includes: determining a second mutual information between the operational feature data and the inference feature data based on the time information in the inference feature data and multiple detection durations in the operational feature data; and generating a fifth verification result based on the second mutual information; wherein the fifth verification result represents the effectiveness of interconnection resource isolation under the preset isolation strategy.

9. The method according to any one of claims 1 to 8, characterized in that, The verification results include verification scores for at least two of the following detection dimensions: video memory residual detection dimension, cache-side channel detection dimension, time-side channel detection dimension, cross-tenant resource interference detection dimension, and data interconnection-side channel detection dimension; the method further includes: Based on the weights corresponding to different detection dimensions, the verification scores in the verification results are weighted and summed to obtain the target isolation security score; wherein, the target isolation security score is used to represent the degree of isolation effectiveness between multiple tenants under the preset isolation strategy.

10. The method according to claim 9, characterized in that, The method further includes: Based on the verification scores corresponding to each detection dimension, determine the corresponding differentiated isolation and hardening strategies; The preset isolation strategy is updated based on the differentiated isolation hardening strategy, and isolation processing is performed between different tenants in the processor based on the updated preset isolation strategy.

11. The method according to claim 10, characterized in that, The step of determining the corresponding differentiated isolation and hardening strategy based on the verification score corresponding to each detection dimension includes at least one of the following: If the verification score corresponding to the memory remnant detection dimension is less than or equal to the first preset security threshold, a first isolation hardening strategy is generated; the first isolation hardening strategy is used to instruct the processor to trigger a zeroing operation on the memory resources after detecting the release of memory resources of any tenant; If the verification score corresponding to the cache side channel detection dimension is less than or equal to the second preset security threshold, a second isolation hardening strategy is generated; the second isolation hardening strategy is used to instruct the processor to perform at least one of the following: enabling multi-instance hardware partition isolation operation, performing cache refresh operation between batches, and reducing batch merging granularity operation; If the verification score corresponding to the time-side channel detection dimension is less than or equal to the third preset security threshold, a third isolation hardening strategy is generated; the third isolation hardening strategy is used to instruct the processor to perform time padding processing that increases the preset inference duration for any tenant's inference request; If the verification score corresponding to the cross-tenant resource interference detection dimension is less than or equal to the fourth preset security threshold, a fourth isolation and hardening strategy is generated; the fourth isolation and hardening strategy is used to instruct the processor to execute at least one of the following: adjusting the multi-instance hardware partition space capacity, limiting the number of concurrent tenants, and enabling computing power service quality control (QoS); If the verification score corresponding to the data interconnect side channel detection dimension is less than or equal to the fifth preset security threshold, a fifth isolation hardening strategy is generated; the fifth isolation hardening strategy is used to instruct the processor to perform at least one of the following: binding dedicated data interconnect resources, enabling interconnect layer traffic filling, and restricting cross-processor communication scheduling alignment.

12. The method according to any one of claims 1 to 8, characterized in that, The method further includes: Upon detecting that the processor's driver or firmware update is complete, the isolation security verification process for the multi-tenant processor is triggered.

13. The method according to claim 2, characterized in that, The method further includes: An asynchronous parallel strategy is adopted to perform the data correlation analysis from at least two of the following detection dimensions: the video memory residual detection dimension, the cache side channel detection dimension, the time side channel detection dimension, the cross-tenant resource interference detection dimension, and the data interconnect side channel detection dimension.

14. An isolation security verification device for a multi-tenant processor, characterized in that, include: The acquisition module is used to acquire, in response to the first tenant performing an inference task, operation feature data of the operation behavior performed by the second tenant on the processor hardware resources used by the first tenant, and inference feature data of the first tenant. Wherein, the first tenant and the second tenant are different tenants isolated in the processor based on a preset isolation strategy; the operation feature data is used to represent the operation behavior; the inference feature data is used to represent the activity status of the first tenant in the inference task; The verification module is used to perform data correlation analysis on the operational feature data and the inference feature data, and generate verification results. The data correlation analysis is used to analyze the data change trends and / or numerical feature correspondences between the operational feature data and the inference feature data. The verification results indicate the isolation effectiveness between multiple tenants under the preset isolation strategy.

15. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the isolated security authentication method for a multi-tenant processor as described in any one of claims 1-13.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the isolation security verification method for a multi-tenant processor as described in any one of claims 1-13.

17. A computer program product, characterized in that, Includes computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device performs the isolated security verification method of a multi-tenant processor as described in any one of claims 1-13.

Images