Device and method for deploying DeepSeek

Abstract

The invention discloses a device and a method for deploying DeepSeek. The device comprises a Weixin H8000 processor, a SW A1 accelerator card, an AI third-party library, an acceleration library, basic software, a compiler, a compiling framework and a DeepSeek large model. The Weixin H8000 processor serves as a core computing unit and is responsible for general computing and resource scheduling; the SW A1 acceleration card provides AI special computing power acceleration; the AI third-party library and the acceleration library optimize the algorithm execution efficiency; the basic software manages system resources; the compiler and the compiling framework are used for adapting software codes to domestic hardware; the DeepSeek large model serves as a top layer AI application, and all component resources of a bottom layer are called; the Weixin H8000 processor and the Shenwei A1 accelerator card construct a heterogeneous computing resource pool to integrate computing power, basic software performs unified scheduling, an AI acceleration library and a compilation framework optimize algorithm execution efficiency, an RDMA high-speed network improves data interaction performance, and a DeepSeek large model cooperatively calls computing power resources of all components to complete an AI computing task. The problem that networked artificial intelligence products lack privacy protection is solved.

Description

Technical Field

[0001] The present invention relates to a device and method for deploying DeepSeek, and belongs to the technical field of artificial intelligence. Background Art

[0002] Currently, there are several deficiencies in the localization and independent control of AI products:

[0003] Incomplete technological autonomy: Although my country continues to promote independent research and development in the field of artificial intelligence, existing artificial intelligence products still rely on foreign technology for some hardware, especially key hardware such as high-performance computing chips and accelerator cards, and cannot achieve comprehensive technological autonomy and control, which poses risks to product security and long-term stability.

[0004] Inadequate security and reliability: AI products also face numerous challenges in terms of security and reliability. First, data security is a core issue, especially in scenarios involving personal privacy and sensitive data. Some AI products still lack sufficient data encryption and privacy protection in networked environments, potentially leading to data leaks. Furthermore, AI computing platforms lack system stability and defense capabilities, making them vulnerable to malicious attacks and technical vulnerabilities. Summary of the Invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art and provide a device and method for deploying DeepSeek to solve the problem of lack of privacy protection in networked artificial intelligence products.

[0006] In order to solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0007] In a first aspect, the present invention provides a device for deploying DeepSeek, comprising: a Weixin H8000 processor, a Shenwei A1 accelerator card, an AI third-party library, an acceleration library, basic software, a compiler, a compilation framework, and a DeepSeek large model;

[0008] The Weixin H8000 processor serves as the core computing unit and is responsible for general computing and resource scheduling;

[0009] The Shenwei A1 accelerator card provides AI-specific computing acceleration;

[0010] The AI ​​third-party library and acceleration library optimize the algorithm execution efficiency;

[0011] The basic software manages system resources;

[0012] The compiler and compilation framework are used to adapt software code to domestic hardware;

[0013] The DeepSeek large model serves as a top-level AI application, calling on the resources of the underlying components;

[0014] The Weixin H8000 processor and the Shenwei A1 accelerator card build a heterogeneous computing resource pool to integrate computing power, unified scheduling of basic software, AI acceleration library and compilation framework optimize algorithm execution efficiency, RDMA high-speed network improves data interaction performance, and the DeepSeek large model collaboratively calls the computing power resources of each component to complete AI computing tasks.

[0015] The aforementioned device that deploys DeepSeek, the Weixin H8000 processor is based on Shenwei's independent SW_64 instruction set microarchitecture, integrates 64 homogeneous SW_64Core4 cores, is equipped with 128MB of third-level shared cache, integrates 8-way 72-bit DDR4 controller and 6-way PCIe4.0 interface, supports multi-way interconnection, hardware multi-threading and hardware virtualization technology, and provides test interface and debugging management interface that comply with IEEE1149.1 standard.

[0016] The aforementioned device for deploying DeepSeek, the Shenwei A1 accelerator card includes the Shenwei A1 accelerator card including the heterogeneous management core MPE and the computing core CPE; the computing core CPE includes the basic software stack, the computing acceleration library and the TCCL communication library, wherein the basic software stack includes the Driver device driver, the Runtime computing runtime library, the Sdaa heterogeneous many-core programming model and the OpenMP parallel programming model, and is also equipped with a heterogeneous many-core programming toolkit consisting of the GDB debugging tool, the Perf performance analysis tool and the Trace run tracking tool; the heterogeneous management core MPE is responsible for the unified scheduling of hardware resources, and the computing core CPE collaborates with the computing acceleration library and the TCCL communication library to complete parallel computing tasks.

[0017] In a second aspect, the present invention provides a method for deploying DeepSeek, which deploys DeepSeek on the DeepSeek deployment device described in the first aspect, comprising:

[0018] S1: Build the hardware development environment for the Weixin H8000 processor;

[0019] S2: Complete the debugging and adaptation of the DeepSeek deployment device's system library, Shenwei A1 accelerator card software stack, Docker container, and Python repository;

[0020] S3: Use the tecoinferencelm inference framework to convert the trained DeepSeek large model into an executable format that adapts to the Shenwei A1 accelerator card hardware architecture;

[0021] S4: Deploy the DeepSeek large model inference backend, use the DeepSeek inference image, mount the model and startup script, and bind the Shenwei A1 accelerator card;

[0022] S5: Start the AI ​​application.

[0023] In the aforementioned method for deploying DeepSeek, step S2 includes:

[0024] S21: Install system libraries, including the gcc compiler, standard c library, cmake tool, and Python 3.9 environment;

[0025] S22: Install the Sunway A1 accelerator card software stack;

[0026] S23: Load the deepseek-32b-a1-server large model backend service Docker image and the chat-nginx Shenwei Q&A platform frontend image;

[0027] S24: Configure the private PIP repository of the device where DeepSeek is deployed to complete full-stack software and hardware adaptation.

[0028] In the aforementioned method for deploying DeepSeek, step S22 includes:

[0029] S221: Obtain the Sunway A1 accelerator card software installation package, including the underlying driver and firmware installation packages, the software development tool installation package, and the AI ​​framework installation package.

[0030] S222: Execute the installation command of each Shenwei A1 accelerator card software installation package to install it;

[0031] S223: Configure environment variables; the environment variables setvars.sh and setvars.sh are installed in the / opt / tecoai path. Execute the source setvars.sh && source setvars.sh command to load the environment variables.

[0032] In the aforementioned method for deploying DeepSeek, the installation commands for each Shenwei A1 accelerator card software installation package in step S222 include:

[0033] The command to install the underlying driver and firmware is: rpm-ivh tecodriver-2.0.1-1.sw_64.rpm;

[0034] The installation command for the software development tool is: rpm-ivh tecotoolkit-2.0.1-1.sw_64.rpm;

[0035] The installation command for the AI ​​framework is: pip3 install *.whl.

[0036] The aforementioned method for deploying DeepSeek, after loading the environment variables in step S223, further includes: executing the teco-smi command to view the basic information of the Shenwei A1 accelerator card. If the basic information of the Shenwei A1 accelerator card can be viewed normally, it indicates that the environment variables are loaded successfully and the software stack is installed successfully.

[0037] In the aforementioned method for deploying DeepSeek, the conversion process in step S3 includes computational graph reconstruction, operator fusion, and precision quantization processing to maximize the computing performance of the accelerator card.

[0038] In the aforementioned method of deploying DeepSeek, in step S5, the AI ​​application methods include ChatBox intelligent assistant and OpenAI paradigm service, and support knowledge base retrieval. Among them, supporting knowledge base retrieval means that the Embedding model optimized by the device deploying DeepSeek vectorizes and stores the knowledge base, uses the A1 accelerator card to accelerate the retrieval process, and combines with the RDMA network to achieve low-latency data transmission. Finally, the DeepSeek large model generates knowledge-enhanced accurate answers based on the retrieval results.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The device deployed by the present invention uses the high-performance server Weixin H8000 as the host, the Shenwei AI chip as the accelerator card, and is equipped with a DeepSeek large model to provide a variety of efficient office functions such as intelligent question and answer, auxiliary programming, academic translation, and data analysis. The device of the present invention achieves comprehensive autonomous control from components to systems, ensuring that it can operate without an Internet connection in the case of local deployment, ensuring data security and privacy, and solving the problem of lack of privacy protection for networked artificial intelligence products. Specific features are as follows:

[0041] Self-developed server:

[0042] This invention adopts an intelligent server based on the Shenwei Weixin H8000 chip, which has the advantages of high computing performance, large-capacity storage, strong expansion, high reliability, and easy management. It is suitable for application scenarios such as artificial intelligence and machine learning, and can meet the high-performance requirements of various industries.

[0043] Nationally produced technical architecture:

[0044] The device deployed by the present invention uses domestically produced hardware and software technology architecture, combined with independently developed core components such as Shenwei A1 card and Weixin H8000 processor, with the characteristics of high performance, stability and security, fully protecting user data privacy and operational security.

[0045] Deploy the DeepSeek deep learning model:

[0046] Based on the DeepSeek deep learning model, the device deployed in this invention has powerful natural language processing capabilities, can quickly understand and process complex user queries, and provide accurate and intelligent answers. With more use, the model will also self-optimize, gradually improving the quality and accuracy of answers.

[0047] Localized smart office solutions:

[0048] The DeepSeek device deployed in this invention supports offline deployment and operates efficiently without relying on an internet connection. Users can safely and quickly complete intelligent question-and-answer, assisted programming, data analysis, and other operations within an intranet environment. It offers a high degree of security and effectively protects the privacy of sensitive data, making it particularly suitable for industries such as government and enterprises that require confidentiality. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 1 is a schematic diagram of the structure of a device for deploying DeepSeek according to Example 1 of the present invention;

[0050] Figure 2 Schematic diagram of the structure of the Weixin H8000 processor according to Example 1 of the present invention;

[0051] Figure 3 This is a diagram of the architecture of the Shenwei A1 accelerator card according to Example 1 of the present invention;

[0052] Figure 4 This is a schematic diagram of basic information of the Shenwei A1 accelerator card terminal according to Example 2 of the present invention;

[0053] Figure 5 This is a schematic diagram of a ChatBox conversation page according to Example 2 of the present invention;

[0054] Figure 6 This is a schematic diagram of the question and answer page of the Shenwei Q&A platform constructed by the OpenAI paradigm service of Example 2 of the present invention. DETAILED DESCRIPTION

[0055] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present application and the specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. Unless there is a conflict, the embodiments of the present application and the technical features in the embodiments can be combined with each other.

[0056] The term "and / or" in this document simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " in this document generally indicates an "or" relationship between the related objects.

[0057] Example 1:

[0058] This embodiment provides a device for deploying Deepseek, such as Figure 1 As shown, the components include: Weixin H8000 processor, Shenwei A1 accelerator card, AI third-party library, acceleration library, basic software, compiler, compilation framework and DeepSeek large model;

[0059] The Weixin H8000 processor serves as the core computing unit and is responsible for general computing and resource scheduling;

[0060] The Shenwei A1 accelerator card provides AI-specific computing acceleration;

[0061] The AI ​​third-party library and acceleration library optimize the algorithm execution efficiency;

[0062] The basic software manages system resources;

[0063] The compiler and compilation framework are used to adapt software code to domestic hardware;

[0064] The DeepSeek large model serves as a top-level AI application, calling on the resources of the underlying components;

[0065] The Weixin H8000 processor and the Shenwei A1 accelerator card build a heterogeneous computing resource pool to integrate computing power, unified scheduling of basic software, AI acceleration library and compilation framework optimize algorithm execution efficiency, RDMA high-speed network improves data interaction performance, and the DeepSeek large model collaboratively calls the computing power resources of each component to complete AI computing tasks.

[0066] The device deployed by the present invention deploys Deepseek to achieve efficient integration of computing, storage, network and AI applications through deep collaboration between software and hardware, and build a nationally produced high-performance intelligent platform.

[0067] The device for deploying Deepseek in this implementation uses an operating system independently developed by China, specifically Tongxin UOS V20 / Kylin / openEuler 2203.

[0068] The Deepseek deployment device implemented in this paper uses the Weixin H8000 processor as the core computing unit, and cooperates with the Shenwei A1 acceleration card to provide heterogeneous computing power. It realizes unified scheduling of hardware resources through a domestic operating system, and relies on the AI ​​acceleration library and compilation framework to optimize the algorithm execution efficiency; the storage system adopts a distributed architecture and RDMA high-speed network protocol to achieve efficient data transmission, and the network layer ensures computing communication through traffic perception and priority scheduling; the DeepSeek large model directly calls the underlying acceleration resources through a dedicated runtime library, and finally the unified management platform dynamically allocates computing, storage and network resources, forming a high-performance intelligent platform with deep collaboration of domestic software and hardware, realizing the efficient integration and resource pooling management of computing, storage, network and AI applications.

[0069] like Figure 2 As shown in the structure diagram of the Shenwei Weixin H8000 processor, the Weixin H8000 processor is based on the SW_64 microarchitecture of Shenwei's independent instruction system. A single chip integrates up to 64 homogeneous SW_64Core4 cores, 128MB of three-level shared cache, 8-way 72-bit DDR4 memory controller (supports ECC), 6-way PCIe4.0 interface, maintenance interface that supports debugging and management, and test interface that complies with the IEEE1149.1 standard; supports multi-way interconnection; supports hardware multi-threading and hardware virtualization.

[0070] Among them, the SW_64Core4 core is a 64-bit homogeneous processing core based on Shenwei's fourth-generation microarchitecture, which is used to: execute general computing instructions, support hardware multi-threading and virtualization, and provide basic computing power support; the 128MB third-level shared cache (LLC) is the last-level shared cache, which is used to: reduce the latency of multi-core access to memory, improve the efficiency of 64-core data sharing, and alleviate the "memory wall" problem; the 8-way 72-bit DDR4 storage controller supports 72 bits (64-bit data + 8-bit ECC) DDR4 memory channels per channel, which is used to: provide a maximum theoretical bandwidth of 204.8GB / s (based on DDR4-3200) to ensure big data throughput; the 6-way PCIe4.0 interface is a x16 channel per channel with a total bandwidth of 96GB / s (bidirectional), which is used to: connect to Shenwei A1 acceleration card (supporting CXL / CCIX protocol), expand RDMA network card (such as 100G RoCEv2), and mount NVMe storage devices.

[0071] The Shenwei A1 accelerator card integrates heterogeneous management cores (MPEs) and computing cores (CPEs). It uses a unified instruction set and execution model for different application characteristics, supports main memory sharing with cache consistency, and reduces the impact of "memory access walls." It supports a variety of collaborative computing technologies between CPEs, including multi-mode data stream transmission, efficient register communication technology, and fast hardware synchronization technology.

[0072] The computing power of a single Shenwei A1 accelerator card can reach 9.37 TFLOPS (FP32), 240 TFLOPS (FP16), 240 TOPS (INT16), and 480 TOPS (INT8), and supports mainstream artificial intelligence frameworks such as Pytorch, PaddlePaddle, and inference engines.

[0073] The basic software layer of the Shenwei A1 accelerator card includes the Driver heterogeneous many-core device driver, the Runtime heterogeneous many-core computing runtime library, the heterogeneous many-core parallel programming model (Sdaa heterogeneous many-core programming model, OpenMP), and the heterogeneous many-core parallel programming toolkit (GDB parallel debugging tool, Perf performance analysis tool, Trace run tracing tool).

[0074] The acceleration library of the Shenwei A1 accelerator card includes computing acceleration libraries (DNN, BLAS, SPARSE, FFT, SCIENCE, etc.) and TCCL communication acceleration libraries. Figure 3 shown.

[0075] Components and functions:

[0076] The accelerator card consists of a heterogeneous management core (MPE) (responsible for task scheduling, resource coordination and host interaction), a computing core (CPE) (performs parallel computing, supports multi-precision operations and inter-core collaboration), a unified cache consistency memory system (realizes data sharing and reduces memory access latency), a multi-mode data stream transmission architecture (supports efficient data exchange between CPEs), register communication and hardware synchronization mechanism (provides low-latency inter-core communication), a basic software stack (including drivers, runtime, programming models and debugging tools, supports task scheduling and development optimization), a computing acceleration library (optimizes DNN / BLAS / FFT and other calculations), and a TCCL communication library (accelerates multi-card collective communication).

[0077] 1. Hardware Architecture

[0078] The accelerator card adopts a heterogeneous many-core architecture and integrates the following core components:

[0079] Management Processing Element (MPE): Responsible for control flow tasks such as task scheduling, data distribution, and exception handling. It interacts with the host, supports cache coherence and main memory sharing, reduces communication overhead between the host and the accelerator card, and coordinates data transfer between the host (CPU) and the accelerator card (such as PCIe DMA).

[0080] Computing Processing Element (CPE): Optimized for high-performance computing, it supports FP32 / FP16 / INT16 / INT8 multi-precision operations, delivering a single-card computing power of 9.37 TFLOPS (FP32), 240 TFLOPS (FP16), 240 TOPS (INT16), and 480 TOPS (INT8). CPEs achieve efficient collaboration through multi-mode data streaming (such as broadcast, point-to-point, and multicast), register communication (low-latency inter-core data exchange), and hardware synchronization technologies (such as atomic operations, semaphores, and fence synchronization).

[0081] Unified Memory Architecture (UMA): Supports global address space and cache coherence. By sharing main memory between the MPE and CPE, it alleviates the "memory wall" problem, reduces data copying, and reduces data transfer overhead. Cache coherence protocols, such as MESI, ensure data consistency.

[0082] High-speed interconnect bus: supports high-bandwidth and low-latency inter-core communication (such as NoC or Crossbar).

[0083] 2. Software Stack

[0084] 1) Driver layer: Provides heterogeneous device management, memory allocation, DMA transfer, etc. for underlying hardware control.

[0085] 2) Runtime: Implements functions such as task parallel scheduling, inter-CPE core synchronization, and memory consistency maintenance, and is used to manage MPE / CPE execution.

[0086] 3) Programming model, used to define parallel computing logic, including SDAA and OpenMP:

[0087] SDAA heterogeneous programming model: Explicitly defines MPE-CPE master-slave collaboration, supporting data parallelism (such as block computing) and task parallelism.

[0088] OpenMP extension: offload host-side tasks to the accelerator card through instruction sets (such as #pragma offload).

[0089] 4) Programming Toolkit:

[0090] GDB parallel debugging tool: supports breakpoint debugging and inter-core status monitoring.

[0091] Perf / Trace tool: Analyze computing hotspots, communication latency, and optimize data flow.

[0092] 5) Acceleration library, used to optimize computing / communication operators, including computing acceleration library and communication acceleration library (TCCL communication library):

[0093] Computing acceleration libraries: DNN (convolution / matrix operations), BLAS (basic linear algebra), FFT (fast Fourier transform), SPARSE sparse computing library, SCIENCE scientific computing acceleration library, UAL unified acceleration library (for Auto automatic code optimization and generation), Librt on-chip runtime library, Kernel computing primitive library, Intrinsics acceleration support library, etc., optimized for CPE architecture.

[0094] TCCL communication library: Optimizes collective operations such as AllReduce and Broadcast between multiple GPUs to improve distributed training efficiency.

[0095] 3. Typical Workflow Examples (Taking AI Reasoning as an Example)

[0096] Phase 1: Task initialization (host side)

[0097] Task loading (application layer call):

[0098] The user calls the accelerator card driver through the PyTorch framework interface on the host and loads the model (such as ResNet50) into the accelerator card shared memory.

[0099] Data preparation:

[0100] Input data (such as images) is transferred to the accelerator card shared memory via PCIe DMA.

[0101] Task division:

[0102] The runtime parses the computational graph and assigns tasks such as convolutional layers and matrix multiplication to the CPE cluster.

[0103] Phase 2: MPE Scheduling and CPE Calculation

[0104] MPE task distribution:

[0105] MPE distributes computing tasks to multiple CPEs and manages data dependencies (such as inter-layer pipelining).

[0106] CPE parallel computing:

[0107] Data stream mode: CPEs exchange intermediate results (such as feature map blocks) through multi-mode data streams.

[0108] Register communication: Neighboring CPEs pass data directly through registers (ultra-low latency).

[0109] Hardware synchronization: Use atomic operations to ensure computation order (such as LayerNorm synchronization).

[0110] Acceleration library optimization: The DNN library calls INT8 quantized convolution kernels to achieve 480 TOPS computing power.

[0111] Phase 3: Result return and performance analysis

[0112] The result is:

[0113] The MPE collects the CPE output and returns the results to the host through consistent memory.

[0114] Performance Analysis:

[0115] Perf tool: Statistics on the utilization rate and computing / communication ratio of each CPE.

[0116] Trace tool: Visualize data flow, analyze and identify data flow bottlenecks (such as DMA latency).

[0117] 4. Performance optimization points

[0118] Debugging optimization: GDB supports breakpoint debugging of inter-core status, Perf analysis of hotspots, use of Trace tools to analyze DMA transfer bottlenecks, and adjustment of data blocking strategies.

[0119] Inter-core communication optimization: Overlap computation and communication through the pipeline communication mode of the TCCL library.

[0120] Computing optimization: mixed precision (FP16 training + INT8 inference) balances accuracy and throughput.

[0121] Summary: This accelerator card leverages heterogeneous hardware collaboration (MPE / CPE), unified memory, efficient inter-core communication, and full-stack software optimization (driver → runtime → acceleration library). It's suitable for scenarios like AI training / inference and scientific computing, significantly improving energy efficiency. A typical task (such as ResNet50 inference) executes in the following order: host data preparation → MPE task scheduling → CPE parallel computing → result return. Cache consistency ensures data correctness throughout the entire process.

[0122] The device deployed with Deepseek in this embodiment supports core dependency libraries such as OpenCV, Numpy, Tokenizer, OnnxRuntime, and Cython, and builds a pip repository based on the device deployed with DeepSeek. Currently, the repository has included 238,968 commonly used Python software packages, of which 653 are related to the Shenwei architecture.

[0123] The device deploying Deepseek in this embodiment supports multiple Deepseek large models, including Deepseek R1-Distill-Qwen-7B, Deepseek R1-Distill-Qwen-32B, Deepseek R1-Distill-llama-70B, etc. If the computing resource environment meets the requirements, Deepseek R1-671B can also be well supported.

[0124] The device deployed with Deepseek in this embodiment can provide multiple functions such as smart office, scientific research and development, voice recognition, text translation, content creation, etc., and is oriented towards the Internet, cloud computing, and local enterprise office services, meeting the needs of diverse, green, and secure computing.

[0125] Example 2:

[0126] Based on the same inventive concept as the first embodiment, this embodiment introduces a method for deploying Deepseek, which deploys Deepseek on the device of the first embodiment, including:

[0127] Hardware and software preparation (AI base construction):

[0128] In one embodiment, a device for deploying DeepSeek relies on a domestic AI base, has features such as out-of-the-box use, containerized management, and private warehouse, and is compatible with the Shenwei A1 software stack to ensure efficient operation in a domestic environment.

[0129] Ready to use right out of the box: Pre-installed AI components ensure immediate use without the need to install complex dependencies. Pre-configured and optimized hardware acceleration libraries improve inference and training efficiency. Adapting to the Chinese software and hardware ecosystem, providing natively compatible drivers and toolchains.

[0130] Containerized management: Based on Docker container management technology, it achieves efficient deployment and operation and maintenance. Compatible with domestic CPU / GPU architectures to ensure stable platform operation.

[0131] Private Repository: A pre-built private pip repository supports enterprise-level data security storage and access control. It can be integrated with the local knowledge base to build a dedicated intelligent question-answering and knowledge retrieval system.

[0132] The specific process includes:

[0133] S1: Hardware Environment Preparation

[0134] Build the hardware development environment for the Sunway processor, including server configuration (see Table 1 for details) and the Sunway A1 accelerator card (see Table 2 for details).

[0135] Table 1 Server configuration of a Deepseek deployment device

[0136]

[0137] Table 2 Parameters of Shenwei A1 accelerator card

[0138]

[0139] S2: Software Environment Preparation

[0140] Ensure that the system libraries, Shenwei A1 accelerator card software stack, Docker container, and Python repository run normally and are deeply adapted on the device where DeepSeek is deployed.

[0141] S21: Install system libraries, including the gcc compiler, standard C library, cmake tool, Python 3.9 environment, etc.

[0142] S22: Install the A1 accelerator card software stack. The installation steps include:

[0143] S221: Obtain the A1 accelerator card software installation package, including the underlying driver and firmware installation package, the software development tool installation package, and the AI ​​framework installation package. See Table 3 for details.

[0144] Table 3. Detailed list of the Shenwei A1 accelerator card software stack

[0145]

[0146] S222: Execute the installation instructions of each Shenwei A1 accelerator card software installation package to install:

[0147] The command to install the underlying driver and firmware is: rpm -ivh tecodriver-2.0.1-1.sw_64.rpm

[0148] The installation command for the software development tool is: rpm -ivh tecotoolkit-2.0.1-1.sw_64.rpm

[0149] The installation command for the AI ​​framework is: pip3 install *.whl

[0150] S223: Configure environment variables:

[0151] The environment variables setvars.sh and setvars.sh are installed in the / opt / tecoai path. Execute the sourcesetvars.sh && source setvars.sh command to load the environment variables.

[0152] Execute the teco-smi command, if it works, you can see the following: Figure 4 The basic information of the Shenwei A1 accelerator card shown indicates that the environment variables were loaded successfully and the software stack was installed successfully.

[0153] S23: Load the large model backend service Docker image named deepseek-32b-a1-server, and load the Shenwei Q&A platform front-end image named chat-nginx.

[0154] S24: Configure the private PIP repository of the device where DeepSeek is deployed to complete the full-stack software and hardware adaptation; the repository currently contains 238,968 commonly used Python software packages, of which 653 are related to the Shenwei architecture.

[0155] S3: Model format conversion and optimization

[0156] The core of the device of Example 1 is the model inference service, which provides high-performance DeepSeek large model inference capabilities and relies on multiple optimization frameworks and runtime libraries to improve inference efficiency.

[0157] tecoinferencelm framework: an inference framework for model optimization that supports efficient operator scheduling and memory management.

[0158] vLLM reasoning framework: Mainly used for reasoning tasks of large models. Adapted to DeepSeek applications in scenarios such as intelligent question answering, code generation, and summary extraction.

[0159] SDAARuntime Library: The heterogeneous computing model runtime library for the Sunway A1 accelerator card maximizes the shielding of underlying hardware features and provides a simple, easy-to-use, and feature-rich API interface to upper-level components. SDAARuntime fully utilizes the parallel data processing capabilities of the Sunway A1 accelerator card, unleashing its powerful computing power.

[0160] The Sunway A1 accelerator card's software stack operates differently from common CUDA software stacks. Large model inference within the Sunway A1's basic software stack requires model conversion using the TecoinferenceLM inference framework. This involves computational graph reconstruction, operator fusion, and precision quantization to maximize the accelerator card's computing performance.

[0161]

[0162] tecoinferencelm is a model optimization inference framework that supports efficient operator scheduling and memory management. It can automatically replace operators with Shenwei A1 high-performance operators, merge redundant nodes, simplify model computation graphs, and optimize memory access patterns.

[0163] S4: Model running

[0164] To adapt to the hardware environment of the Shenwei AI base, the DeepSeek model needs to be converted, optimized, and performance-tuned to improve inference efficiency. This embodiment uses the performance tuning method of the SDAA C programming model to maximize the hardware resources of the Shenwei A1 accelerator card's heterogeneous parallel computing platform to achieve optimal performance programming.

[0165] Performance measurement: Use the performance sampling tools and performance analysis tools supported by the SDAA C programming model to evaluate the model's inference performance, analyze key indicators such as inference latency, throughput, and power consumption, optimize computing resource allocation, and ensure that the inference effect meets application requirements.

[0166] Performance Tuning: Use some compilation optimization methods supported by the SDAA C programming model, combined with quantization technology, to reduce computing costs and improve inference speed. Optimize memory management to reduce GPU / CPU memory usage and improve inference stability.

[0167] Operator optimization: Combined with the Shenwei A1 accelerator card, develop efficient parallel algorithms or optimize the performance of existing algorithms.

[0168] The specific steps include:

[0169] Deploy the DeepSeek large model inference backend, use the DeepSeek inference image, mount the model and startup script, and bind the Shenwei A1 accelerator card device to start the backend service.

[0170]

[0171] The backend core service is started through a Python script, which combines the vLLM and tecoinferencelm inference frameworks to run the DeepSeek large model and relies on the Shenwei A1 accelerator card for heterogeneous computing acceleration.

[0172]

[0173] S5: Start AI application running

[0174] The method of this embodiment deploys the DeepSeek large model, and the device of embodiment 1 supports multiple AI application modes, mainly including ChatBox intelligent assistant and OpenAI paradigm service, and supports knowledge base retrieval.

[0175] The support for knowledge base retrieval refers to the present invention deploying the device-optimized Embedding model of DeepSeek to vectorize and store the knowledge base, using the A1 accelerator card to accelerate the retrieval process, and combining the RDMA network to achieve low-latency data transmission. Ultimately, the DeepSeek large model generates accurate answers with knowledge enhancement based on the retrieval results.

[0176] Among them, ChatBox intelligent assistant is an interactive dialogue system based on the DeepSeek large model, providing natural language question and answer, task planning and multi-round dialogue capabilities, suitable for scenarios such as enterprise intelligent customer service and knowledge consulting; OpenAI paradigm service refers to providing model reasoning capabilities through a standard REST API interface, which is compatible with the OpenAI protocol (such as / v1 / chat / completions), making it easier for developers to integrate into the existing AI application ecosystem. Both support enhancing answer accuracy through knowledge base retrieval.

[0177] ChatBox Intelligent Assistant: It runs in a remote environment and can perform debugging and multi-round dialogue interaction. It combines with a private knowledge base to provide customized knowledge questions and answers to improve business adaptability. ChatBox dialogue page, such as Figure 5 shown.

[0178] OpenAI Paradigm Service: Combine OpenAI API with front-end development to build the Shenwei Q&A platform, such as Figure 6 As shown. The platform can be deployed on local servers, effectively ensuring data security, preventing sensitive information leaks, and ensuring the full protection of enterprise and user data privacy. The Shenwei Q&A platform is an application with standardized interfaces, openness, support for local deployment, and integrated knowledge base services.

[0179] The front end of the Shenwei Q&A platform is a Web interactive UI built on the Nginx + HTML / JS framework. It provides page services as a WebServer and proxies to the backend port 8002 for inference requests. Figure 6 This is the dialogue page of Shenwei Q&A platform.

[0180]

[0181] At this point, the process of deploying DeepSeek is complete.

[0182] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0183] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0184] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0185] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0186] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A device for deploying DeepSeek, characterized in that the components include: Weixin H8000 processor, Shenwei A1 accelerator card, AI third-party library, acceleration library, basic software, compiler, compilation framework and DeepSeek large model; The Weixin H8000 processor serves as the core computing unit and is responsible for general computing and resource scheduling; The Shenwei A1 accelerator card provides AI-specific computing acceleration; The AI ​​third-party library and acceleration library optimize the algorithm execution efficiency; The basic software manages system resources; The compiler and compilation framework are used to adapt software code to domestic hardware; The DeepSeek large model serves as a top-level AI application, calling on the resources of the underlying components; The Weixin H8000 processor and the Shenwei A1 accelerator card build a heterogeneous computing resource pool to integrate computing power, unified scheduling of basic software, AI acceleration library and compilation framework optimize algorithm execution efficiency, RDMA high-speed network improves data interaction performance, and the DeepSeek large model collaboratively calls the computing power resources of each component to complete AI computing tasks.

2. The DeepSeek deployment device according to claim 1, characterized in that: The Weixin H8000 processor is based on Shenwei's independent SW_64 instruction set microarchitecture, integrates 64 homogeneous SW_64Core4 cores, is equipped with 128MB of third-level shared cache, integrates 8-way 72-bit DDR4 controllers and 6-way PCIe4.0 interfaces, supports multi-way interconnection, hardware multi-threading and hardware virtualization technology, and provides test interfaces and debugging management interfaces that comply with the IEEE1149.1 standard.

3. The DeepSeek deployment device according to claim 1, characterized in that: The Shenwei A1 accelerator card includes a heterogeneous management core MPE and a computing core CPE; the computing core CPE includes a basic software stack, a computing acceleration library and a TCCL communication library, wherein the basic software stack includes a Driver device driver, a Runtime computing runtime library, a Sdaa heterogeneous many-core programming model and an OpenMP parallel programming model, and is also equipped with a heterogeneous many-core programming toolkit consisting of a GDB debugging tool, a Perf performance analysis tool and a Trace run tracking tool; the heterogeneous management core MPE is responsible for unified scheduling of hardware resources, and the computing core CPE collaborates with the computing acceleration library and the TCCL communication library to complete parallel computing tasks.

4. A method for deploying DeepSeek, deploying DeepSeek on the DeepSeek deployment device according to any one of claims 1 to 3, characterized in that: include: S1: Build the hardware development environment for the Weixin H8000 processor; S2: Complete the debugging and adaptation of the DeepSeek deployment device's system library, Shenwei A1 accelerator card software stack, Docker container, and Python repository; S3: Use the tecoinferencelm inference framework to convert the trained DeepSeek large model into an executable format that adapts to the Shenwei A1 accelerator card hardware architecture; S4: Deploy the DeepSeek large model inference backend, use the DeepSeek inference image, mount the model and startup script, and bind the Shenwei A1 accelerator card; S5: Start the AI ​​application.

5. The method for deploying DeepSeek according to claim 4, wherein: Step S2 includes: S21: Install system libraries, including the gcc compiler, standard c library, cmake tool, and Python 3.9 environment; S22: Install the Sunway A1 accelerator card software stack; S23: Load the deepseek-32b-a1-server large model backend service Docker image and the chat-nginx Shenwei Q&A platform frontend image; S24: Configure the private PIP repository of the device where DeepSeek is deployed to complete full-stack software and hardware adaptation.

6. The method for deploying DeepSeek according to claim 5, wherein: Step S22 includes: S221: Obtain the Sunway A1 accelerator card software installation package, including the underlying driver and firmware installation packages, the software development tool installation package, and the AI ​​framework installation package. S222: Execute the installation command of each Shenwei A1 accelerator card software installation package to install it; S223: Configure environment variables; The environment variables setvars.sh and setvars.sh are installed in the / opt / tecoai path. Execute the sourcesetvars.sh && source setvars.sh command to load the environment variables.

7. The method according to claim 6, characterized in that The installation commands for each Shenwei A1 accelerator card software installation package in step S222 include: The command to install the underlying driver and firmware is: rpm-ivh tecodriver-2.0.1-1.sw_64.rpm; The installation command for the software development tool is: rpm-ivh tecotoolkit-2.0.1-1.sw_64.rpm; The installation command for the AI ​​framework is: pip3 install *.whl.

8. The method according to claim 6, characterized in that After loading the environment variables in step S223, the process also includes: executing the teco-smi command to view the basic information of the Shenwei A1 accelerator card. If the basic information of the Shenwei A1 accelerator card can be viewed normally, it indicates that the environment variables are loaded successfully and the software stack is installed successfully.

9. The method according to claim 4, characterized in that The conversion process in step S3 includes computational graph reconstruction, operator fusion, and precision quantization processing to maximize the computing performance of the accelerator card.

10. The DeepSeek deployment device according to claim 5, characterized in that: In step S5, AI applications include ChatBox intelligent assistant and OpenAI paradigm service, and support knowledge base retrieval. Among them, OpenAI paradigm service builds the Shenwei question and answer platform; supporting knowledge base retrieval means deploying DeepSeek's device-optimized Embedding model to vectorize and store the knowledge base, using the A1 accelerator card to accelerate the retrieval process, and combining the RDMA network to achieve low-latency data transmission. Finally, the DeepSeek large model generates knowledge-enhanced accurate answers based on the retrieval results.

Images