Model training environment deployment method and system based on TIPU card

Abstract

The invention discloses a model training environment deployment method and system based on a TIPU card, and the method comprises the steps: guaranteeing that the TIPU card is correctly installed on a computing power server, and enabling management ports and storage ports of a storage server, the computing power server and the TIPU card to be connected with a switch; when the computing power server is powered on to start the system, a trusted root program of the TIPU card is started, a system environment is initialized, configuration including an IP address, a host name and a shared storage link is completed, and it is confirmed that management networks and storage networks of all devices are normally connected; loading security firmware on the TIPU card, wherein the security firmware comprises an operating system kernel and a driving program; on the basis of the security firmware, software and tools required by model training are deployed. According to the method, the data security in the model training process can be ensured, and data leakage and malicious tampering of the model are prevented.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method and system for deploying a model training environment based on a TIPU card. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, the security and reliability of model training environments are receiving increasing attention. Existing deployment methods for model training environments pose security risks, such as unauthorized access to sample data leading to data leakage, malicious tampering with training models leading to erroneous training results, and even directly impacting the effectiveness of model applications in business systems. Therefore, building a secure and reliable model training environment has become a pressing issue. Summary of the Invention

[0003] In view of this, the present application provides a method and system for deploying a model training environment based on a TIPU card.

[0004] This application discloses a TIPU card-based model training environment deployment method, which includes:

[0005] Ensure that the TIPU card is correctly installed on the computing server, and the management and storage ports of the storage server, computing server, and TIPU card are connected to the switch;

[0006] When the computing server is powered on and the system is started, the TIPU card trusted root program starts and initializes the system environment, completing the configuration including the IP address, host name, and shared storage link, and confirming that the management network and storage network of all devices are properly connected;

[0007] Load the security firmware on the TIPU card, which includes the operating system kernel and drivers; based on the security firmware, deploy the software and tools required for model training.

[0008] Furthermore, in the deployed model training environment, the administrator, as the sample owner, uploads the sample through the trusted microenvironment center of the TIPU card, and the sample is encrypted and saved in the shared storage directory corresponding to the storage server.

[0009] Furthermore, the administrator uploads samples through the sample upload function provided by the trusted microenvironment center of the TIPU card. The trusted microenvironment center uses the encryption API to call the encryption interface of the back-end component TIPU gateway to complete sample encryption; the user, as the model owner, applies for sample data in the sample management list through the trusted microenvironment of the TIPU card, and generates a sample data use application work order. The background calls the TIPU gateway API interface to realize administrator approval, thereby automatically completing the sample use authorization, and the decrypted sample data is mounted to the computing power server in the form of file storage unloading through the TIPU card storage port. The decrypted sample data can only be seen on the computing power server, and the decrypted data cannot be seen on the storage server.

[0010] Furthermore, a trusted connection method for the computing power server used for model training is provided. The model owner logs in to the computing power server through the trusted connection method to perform model training. During the training process, the trusted microenvironment center of the TIPU card is responsible for monitoring the security status of the system to prevent malicious attacks and data leakage.

[0011] Furthermore, after the model training is completed, a security assessment of the training environment is conducted. The security assessment includes system vulnerability scanning and model security analysis. Based on the assessment results, the training environment is optimized and adjusted to improve the security and credibility of the model training.

[0012] Furthermore, during the model training process, the model's sample data is compared with the training data to check whether the original data is lost, duplicated or damaged, so as to ensure the integrity of the original data in each link of collection, transmission and storage.

[0013] Furthermore, during the data preprocessing phase of model training, multi-level data integrity verification is implemented through the TIPU card trusted microenvironment center, including:

[0014] Original data verification stage: When the sample owner uploads the original data to the storage server through the encryption API, the trusted microenvironment center synchronously calculates the first hash value H1 of the original data and stores H1 together with the encrypted data in the secure storage area of the TIPU card;

[0015] Training data generation stage: When the TIPU gateway decrypts the sample data and generates the training data set, the trusted microenvironment center calculates the second hash value H2 of the decrypted data in real time, and compares the consistency of the second hash value H2 with the first hash value H1 through the built-in cryptographic coprocessor of the TIPU card;

[0016] Training data loading phase: When the computing server mounts the decrypted sample through the storage port, the trusted microenvironment center calculates the third hash value H3 of the real-time loaded data based on the encryption and decryption module built into the TIPU card, and performs a secondary verification with the second hash value H2;

[0017] Among them, when a difference occurs in the hash value comparison at any level, the trusted microenvironment center performs the following operations: immediately terminate the storage port data transmission of the current training task; record the abnormal data fingerprint in the audit log; automatically generate an abnormal work order containing the comparison results of the first hash value H1, the second hash value H2, and the third hash value H3 and push it to the administrator; trigger a security assessment of the training environment, and optimize and adjust the training environment based on the assessment results to achieve traceability verification.

[0018] The present application also discloses a TIPU card-based model training environment deployment system, which implements the above-mentioned TIPU card-based model training environment deployment method, comprising:

[0019] The first deployment module is used to ensure that the TIPU card has been correctly installed on the computing server, and the management ports and storage ports of the storage server, computing server, and TIPU card have been connected to the switch;

[0020] The initialization module is used to start the TIPU card trusted root program when the computing server is powered on to start the system, initialize the system environment, complete the configuration including IP address, host name, shared storage link, and confirm that the management network and storage network of all devices are properly connected;

[0021] The second deployment module is used to load the security firmware on the TIPU card. The security firmware includes the operating system kernel and drivers; based on the security firmware, the software and tools required for model training are deployed.

[0022] Due to the adoption of the above technical solution, this application has the following advantages:

[0023] 1. This application uses the hardware-level data security protection measures provided by the TIPU card to ensure the security of data during model training and prevent data leakage and malicious tampering of the model.

[0024] 2. This application implements security management of the entire data life cycle to ensure the security and privacy protection of model training data.

[0025] 3. This application aims to improve the manageability and adaptability of the model training environment to cope with ever-changing cybersecurity threats.

[0026] 4. This application continuously improves the security of the model training environment through security assessment and optimization, and adapts to ever-changing network security threats. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0028] Figure 1 This is a network topology diagram of the model training environment in an embodiment of the present application;

[0029] Figure 2 This is a schematic diagram of the use and management of model training sample data in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The present application is further described with reference to the accompanying drawings and embodiments. The embodiments described are only a part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field should fall within the scope of protection of the embodiments of the present application.

[0031] In response to the problems existing in the above-mentioned background technology, this application aims to solve the following technical problems, including: how to ensure the security and credibility of the model training environment, prevent the leakage of original sample data, training result data and malicious tampering of the model; how to achieve full life cycle security management of data that is available but invisible during the model training process; how to improve the manageability and adaptability of the model training environment to cope with ever-changing network security threats.

[0032] See also Figure 1 and Figure 2 , the present application provides an embodiment of a method for deploying a model training environment based on a TIPU card, which includes:

[0033] Ensure that the TIPU card is correctly installed on the computing server, and the management and storage ports of the storage server, computing server, and TIPU card are connected to the switch;

[0034] When the computing server is powered on and the system is started, the TIPU card trusted root program starts and initializes the system environment, completing the configuration including the IP address, host name, and shared storage link, and confirming that the management network and storage network of all devices are properly connected;

[0035] Load the security firmware on the TIPU card, which includes the operating system kernel and drivers; based on the security firmware, deploy the software and tools required for model training.

[0036] Figure 1Figure 2 shows a network diagram of a trusted model training environment. The TIPU card is installed on the computing server. All servers are connected to the management port and the storage port, and the TIPU card accesses data on the storage server through the storage port. The TIPU card serves as the core security component of the model training environment, leveraging its universal DPU functionality and built-in root of trust to ensure security and trustworthiness during system startup and operation. Secure firmware, including the operating system kernel and drivers, is deployed on the TIPU card to manage system resources and ensure data security and integrity. Computing servers, storage servers, and switches are prepared, and the network topology is planned to ensure efficient and stable operation of the model training environment. Based on the secure firmware, the software and tools required for model training, such as deep learning frameworks and optimization algorithms, are deployed. During model training, the TIPU card's trusted microenvironment center monitors the system's security status to prevent malicious attacks and data leaks. After model training is complete, a security assessment of the training environment is conducted, including system vulnerability scanning and model security analysis. Optimization and adjustments are then made based on the assessment results.

[0037] Optionally, in a deployed model training environment, the administrator, as the sample owner, uploads the sample through the trusted microenvironment center of the TIPU card, and the sample is encrypted and saved in the shared storage directory corresponding to the storage server.

[0038] Optionally, the administrator uploads samples through the sample upload function provided by the trusted microenvironment center of the TIPU card. The trusted microenvironment center uses the encryption API to call the encryption interface of the back-end component TIPU gateway to complete sample encryption; the user, as the model owner, applies for sample data in the sample management list through the trusted microenvironment of the TIPU card, and generates a sample data use application work order. The background calls the TIPU gateway API interface to implement administrator approval, thereby automatically completing sample use authorization, and the decrypted sample data is mounted to the computing power server in the form of file storage unloading through the TIPU card storage port. The decrypted sample data can only be seen on the computing power server, and the decrypted data cannot be seen on the storage server.

[0039] Optionally, a trusted connection method for the computing power server is provided for model training. The model owner logs in to the computing power server through the trusted connection method to perform model training. During the training process, the trusted microenvironment center of the TIPU card is responsible for monitoring the security status of the system to prevent malicious attacks and data leakage.

[0040] Optionally, after model training is completed, a security assessment of the training environment is conducted. The security assessment includes system vulnerability scanning and model security analysis. Based on the assessment results, the training environment is optimized and adjusted to improve the security and credibility of model training.

[0041] Optionally, during the model training process, the model's sample data is compared with the training data to check whether the original data is lost, duplicated, or damaged, so as to ensure the integrity of the original data in each link of collection, transmission, and storage.

[0042] Optionally, during the data preprocessing phase of model training, multi-level data integrity verification is implemented through the TIPU card trusted microenvironment center, including:

[0043] Original data verification stage: When the sample owner uploads the original data to the storage server through the encryption API, the trusted microenvironment center synchronously calculates the first hash value H1 of the original data and stores H1 together with the encrypted data in the secure storage area of the TIPU card;

[0044] Training data generation stage: When the TIPU gateway decrypts the sample data and generates the training data set, the trusted microenvironment center calculates the second hash value H2 of the decrypted data in real time, and compares the consistency of the second hash value H2 with the first hash value H1 through the built-in cryptographic coprocessor of the TIPU card;

[0045] Training data loading phase: When the computing server mounts the decrypted sample through the storage port, the trusted microenvironment center calculates the third hash value H3 of the real-time loaded data based on the encryption and decryption module built into the TIPU card, and performs a secondary verification with the second hash value H2;

[0046] Among them, when a difference occurs in the hash value comparison at any level, the trusted microenvironment center performs the following operations: immediately terminate the storage port data transmission of the current training task; record the abnormal data fingerprint in the audit log; automatically generate an abnormal work order containing the comparison results of the first hash value H1, the second hash value H2, and the third hash value H3 and push it to the administrator; trigger a security assessment of the training environment, and optimize and adjust the training environment based on the assessment results to achieve traceability verification.

[0047] Use the TIPU card as the core security component of the model training environment, leveraging its general-purpose DPU functionality and built-in root of trust to ensure security and trustworthiness during system startup and operation. Deploy secure firmware on the TIPU card, including the operating system kernel, drivers, etc., to manage system resources and ensure data security and integrity. Prepare computing servers, storage servers, and switch devices, and plan the network topology to ensure efficient and stable operation of the model training environment. Based on the secure firmware, deploy the software and tools required for model training, such as deep learning frameworks and optimization algorithms. During the model training process, use the TIPU card's trusted microenvironment center to monitor the system's security status to prevent malicious attacks and data leaks. After model training is completed, conduct a security assessment of the training environment, including system vulnerability scanning and model security analysis, and optimize and adjust based on the assessment results.

[0048] The present application also provides an embodiment of a TIPU card-based model training environment deployment system, which implements the TIPU card-based model training environment deployment method described in the above embodiment, comprising:

[0049] The first deployment module is used to ensure that the TIPU card has been correctly installed on the computing server, and the management ports and storage ports of the storage server, computing server, and TIPU card have been connected to the switch;

[0050] The initialization module is used to start the TIPU card trusted root program when the computing server is powered on to start the system, initialize the system environment, complete the configuration including IP address, host name, shared storage link, and confirm that the management network and storage network of all devices are properly connected;

[0051] The second deployment module is used to load the security firmware on the TIPU card. The security firmware includes the operating system kernel and drivers; based on the security firmware, the software and tools required for model training are deployed.

[0052] For the sake of convenience, this application provides a more specific embodiment:

[0053] Within a certain enterprise, the model training environment is deployed using the method provided in this application.

[0054] First, ensure that the TIPU card (Trusted Intelligent Interconnect Card) has been correctly installed on the computing server hardware, the management port and storage port of the storage server, computing server, and TIPU card have been connected to the switch, and all devices are powered on and running.

[0055] Then, when the computing power server is powered on to start the system, the TIPU card trusted root program (the role of the trusted root program here is to initialize and detect the hardware platform to ensure that the hardware environment is trustworthy. The hardware here includes CPU, memory, etc.) starts and initializes the system environment, completing the configuration including IP address, host name, shared storage link, and confirming that the management network and storage network of all devices are connected normally.

[0056] Next, the secure firmware on the TIPU card is loaded, including components such as the operating system kernel and drivers. Based on the secure firmware, the software and tools required for model training are deployed, such as the TensorFlow deep learning framework and the Adam optimization algorithm.

[0057] In the deployed model training environment, the sample owner uploads the sample through the trusted microenvironment center of the TIPU card, and the sample is encrypted and saved in the shared storage directory corresponding to the storage server. By saving the sample in the shared storage directory, it can be ensured that the sample can be safely stored by the storage server. The storage server here is a resource under user control and management, so it can prevent unauthorized access to the sample data. Taking the captured video as an example, the video is converted into multiple pictures by extracting frames. Each picture contains a specific target object, such as a vehicle, pedestrian, building, etc. Different pictures show different postures and features of the target object from different angles. After analyzing and identifying a large number of such samples, the model system can form the feature library data of the object, which can help users identify the target object. For specific fields, samples cannot be easily accessed and used.

[0058] The model owner applies for sample use through the TIPU card's trusted microenvironment, and the administrator authorizes sample use through the TIPU card's trusted microenvironment center. At this point, the encrypted sample data is decrypted and mounted to the computing server via TIPU using virtiofs. This data is only visible on the computing server and not on the storage server. Because the data is encrypted and uploaded to the shared storage directory of the storage server, it is invisible to the storage server without decryption. It is visible on the computing server because it is decrypted and provides a trusted connection method for the computing server used for model training (such as VNC Viewer). The model owner logs in to the computing server through this trusted connection to conduct model training. VNC stands for Virtual Network Computing, and VNC Viewer is client software used to view and operate remote computer desktops. Virtiofs is a file system based on the virtio framework, which is used to efficiently implement file system sharing and interaction between clients and hosts in a virtualized environment. Here, the TIPU and the computing power server establish a data connection and communication through the PCIe interface.

[0059] During training, the TIPU card's trusted microenvironment center (mainly used in scenarios with multiple TIPU cards) monitors the security status of the system (the system refers to the embedded operating system running within the TIPU card) to prevent malicious attacks and data leaks. After model training is complete, a security assessment of the training environment is conducted, including system vulnerability scanning and model security analysis. Based on the assessment results, the training environment is optimized and adjusted (this optimization and adjustment can be achieved using existing technologies) to improve the security and credibility of model training.

[0060] The model sample is uploaded by the sample owner and encrypted and saved to the storage server through the trusted micro-management center of the TIPU card. During model training, the applicant applies for the sample and is authorized by the sample administrator to retrieve and use it.

[0061] The key points and protection points of this application are mainly concentrated in the following aspects:

[0062] Security function integration of TIPU card: The TIPU card not only has the general DPU (Data Processing Unit, a high-performance chip specially designed to process data traffic and network functions in data centers) functions, but also integrates security functions such as encryption and decryption engines and built-in trusted roots. This is a key technical point to ensure the security of the model training environment.

[0063] Design and deployment of secure firmware: The secure firmware on the TIPU card, including the operating system kernel and drivers, is an important technical measure to protect the model training environment from malicious attacks.

[0064] Monitoring and protection of the model training environment: Using the trusted microenvironment center of the TIPU card to monitor the model training process and prevent data leakage and malicious attacks is the core technology of this application to protect the security of model training.

[0065] Security Assessment: During model training, security assessment and optimization are key steps to ensure model performance and data security. The following are the key steps:

[0066] Data Verification: Compare the original data with the training data and, by calculating hash values, strictly check for data loss, duplication, or corruption to ensure data integrity at every stage of collection, transmission, and storage. Privacy Assessment: Detailed review of the data to determine whether it contains sensitive information, and a comprehensive assessment of encryption measures used during data storage and use to effectively prevent data leakage risks. Distribution Analysis: In-depth analysis of the distribution of training data, carefully investigating data biases, to avoid overfitting or insufficient generalization of the model due to uneven data distribution. Training Evaluation: Algorithm Review: Comprehensively inspect deep learning frameworks and optimization algorithms to identify security vulnerabilities or backdoors, focusing on algorithms vulnerable to gradient attacks. Process Monitoring: Real-time dynamic monitoring of CPU and GPU usage and memory usage, closely monitoring the update process of model parameters, and promptly determining whether there are any abnormal fluctuations. Sample Testing: Input adversarial samples into the model and carefully observe the model's response to accurately determine the model's robustness.

[0067] Optimization mechanism: During the model training process, security assessment is only the basis. What is more critical is how to perform targeted optimization based on the assessment results to improve the security and stability of the model. The following is the specific optimization process: Data processing: For unbalanced data, use oversampling and undersampling to balance the data distribution, enhance data diversity through operations such as rotation and scaling, encrypt sensitive data, and desensitize sensitive information that does not need to be retained; Algorithm optimization: Update and reinforcement: Timely update deep learning frameworks and algorithms to reinforce vulnerable algorithms; Robust training: Use adversarial training technology to allow the model to learn to resist adversarial sample attacks; Monitoring optimization: Improve the system: Build a more complete real-time monitoring system, monitor loss functions, accuracy and other indicators, set thresholds and alarms; By optimizing the anomaly detection algorithm, immediately suspend training and analyze and process anomalies when they are found. Ultimately, it ensures that the model training environment can continue to adapt to new security threats, which is also an important technical point for this application to provide long-term security guarantees.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit them. Although the present application has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present application can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present application should be included in the scope of protection of the claims of the present application.

Claims

1. A TIPU card-based model training environment deployment method, characterized in that: include: Ensure that the TIPU card is correctly installed on the computing server, and the management and storage ports of the storage server, computing server, and TIPU card are connected to the switch; When the computing server is powered on and the system is started, the TIPU card trusted root program starts and initializes the system environment, completing the configuration including the IP address, host name, and shared storage link, and confirming that the management network and storage network of all devices are properly connected; Load the security firmware on the TIPU card, which includes the operating system kernel and drivers; based on the security firmware, deploy the software and tools required for model training.

2. The TIPU card-based model training environment deployment method according to claim 1, characterized in that: In the deployed model training environment, the administrator, as the sample owner, uploads the sample through the trusted microenvironment center of the TIPU card. The sample is encrypted and saved in the shared storage directory corresponding to the storage server.

3. The TIPU card-based model training environment deployment method according to claim 2, characterized in that: The administrator uploads samples through the sample upload function provided by the trusted microenvironment center of the TIPU card. The trusted microenvironment center uses the encryption API to call the encryption interface of the back-end component TIPU gateway to complete sample encryption; as the model owner, the user applies for sample data in the sample management list through the trusted microenvironment of the TIPU card, and generates a sample data use application work order. The back-end calls the TIPU gateway API interface to implement administrator approval, thereby automatically completing sample use authorization, and the decrypted sample data is mounted to the computing power server in the form of file storage unloading through the TIPU card storage port. The decrypted sample data can only be seen on the computing power server, and the decrypted data cannot be seen on the storage server.

4. The TIPU card-based model training environment deployment method according to claim 3, characterized in that: Provides a trusted connection method for computing power servers used for model training. Model owners log in to the computing power server through the trusted connection method to perform model training. During the training process, the trusted microenvironment center of the TIPU card is responsible for monitoring the security status of the system to prevent malicious attacks and data leakage.

5. The TIPU card-based model training environment deployment method according to claim 4, characterized in that: After model training is completed, a security assessment of the training environment is conducted. The security assessment includes system vulnerability scanning and model security analysis. Based on the assessment results, the training environment is optimized and adjusted to improve the security and credibility of model training.

6. The TIPU card-based model training environment deployment method according to claim 4, characterized in that: During the model training process, the model's sample data is compared with the training data to check whether the original data is lost, duplicated, or damaged, so as to ensure the integrity of the original data in each link of collection, transmission, and storage.

7. The TIPU card-based model training environment deployment method according to claim 4, characterized in that: During the data preprocessing phase of model training, multi-level data integrity verification is implemented through the TIPU card trusted microenvironment center, including: Original data verification stage: When the sample owner uploads the original data to the storage server through the encryption API, the trusted microenvironment center synchronously calculates the first hash value H1 of the original data and stores H1 together with the encrypted data in the secure storage area of the TIPU card; Training data generation stage: When the TIPU gateway decrypts the sample data and generates the training data set, the trusted microenvironment center calculates the second hash value H2 of the decrypted data in real time, and compares the consistency of the second hash value H2 with the first hash value H1 through the built-in cryptographic coprocessor of the TIPU card; Training data loading phase: When the computing server mounts the decrypted sample through the storage port, the trusted microenvironment center calculates the third hash value H3 of the real-time loaded data based on the encryption and decryption module built into the TIPU card, and performs a secondary verification with the second hash value H2; Among them, when a difference occurs in the hash value comparison at any level, the trusted microenvironment center performs the following operations: immediately terminate the storage port data transmission of the current training task; record the abnormal data fingerprint in the audit log; automatically generate an abnormal work order containing the comparison results of the first hash value H1, the second hash value H2, and the third hash value H3 and push it to the administrator; trigger a security assessment of the training environment, and optimize and adjust the training environment based on the assessment results to achieve traceability verification.

8. A TIPU card-based model training environment deployment system, which implements the TIPU card-based model training environment deployment method according to any one of claims 1 to 7, characterized in that: include: The first deployment module is used to ensure that the TIPU card has been correctly installed on the computing server, and the management ports and storage ports of the storage server, computing server, and TIPU card have been connected to the switch; The initialization module is used to start the TIPU card trusted root program when the computing server is powered on to start the system, initialize the system environment, complete the configuration including IP address, host name, shared storage link, and confirm that the management network and storage network of all devices are properly connected; The second deployment module is used to load the security firmware on the TIPU card. The security firmware includes the operating system kernel and drivers; based on the security firmware, the software and tools required for model training are deployed.

Images