Method, device and system for fast inference of large language model for smart phones

By using the collaborative work of heterogeneous computing units in smartphones and automatically selecting inference strategies, the problem of limited inference performance of smartphone large language models is solved, and efficient utilization of computing resources and rapid response is achieved.

CN118446321BActive Publication Date: 2024-11-29SHANGHAI JIAOTONG UNIV
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202410689239.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-30
Publication Date
2024-11-29
Estimated Expiration
2044-05-30

AI Technical Summary

Technical Problem

Smartphones are limited by their limited computing power and bandwidth in terms of large-language model inference, resulting in a reduction in model intelligence capabilities. Current technologies such as sparse inference fail to fully utilize heterogeneous hardware features and cannot effectively improve inference performance.

Method used

By building a detection system, collecting device information, automatically selecting inference strategies, and using heterogeneous computing units (CPU, GPU, NPU) to work efficiently in different stages, including scheduling matrix multiplication to NPU in the pre-filling stage, and using CPU or The GPU performs sparse calculations and records resource usage information to dynamically adjust the strategy.

Benefits of technology

Significantly improves inference speed and performance, reduces computing latency, provides faster response time, and is suitable for a variety of hardware architectures, enabling adaptive optimization strategies and dynamic performance monitoring.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118446321B_ABST
    Figure CN118446321B_ABST
Patent Text Reader

Abstract

The present invention provides a method, device and system for fast reasoning of a large language model of a smart phone, including step S1: constructing a detection system, using the detection system to collect information of a device to be tested; the information of the device to be tested includes hardware configuration information, model configuration information and user demand information; step S2: enabling the detection system to automatically select a corresponding reasoning strategy according to the information of the device to be tested; step S3: loading parameters of a large language model, initializing the reasoning environment, and starting the reasoning process; the reasoning process includes a pre-filling stage and a decoding stage; step S4: generating and returning the reasoning result to form a result report; adjusting the reasoning strategy after obtaining user feedback, and optimizing the reasoning process. The present invention makes full use of heterogeneous computing units such as CPU, GPU and NPU in a smart phone through an intelligent scheduling algorithm, and optimizes the use efficiency of computing resources; compared with traditional methods, the present invention can significantly improve the reasoning speed and performance.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of large language model prediction, and in particular to a large language model fast inference method and system for a smart phone. Background Art

[0002] Large Language Models (LLMs) have significantly improved our daily lives and revolutionized our work environments with their superior text understanding and generation capabilities. Current state-of-the-art LLMs, such as GPT-4 and Claude-3, are typically hosted in data centers equipped with state-of-the-art GPUs (e.g., NVIDIA H100), which provide extensive high-bandwidth memory and have computational power of thousands of teraflops. At the same time, there is a growing trend that LLMs are moving toward deployment on smartphones, transforming them into intelligent personal assistants. This shift aims to fully maintain privacy by avoiding the transmission of private data to cloud services. However, despite the widespread use of smartphones, they are still unable to cope with the complex requirements of LLM reasoning, mainly due to their limited computational power and bandwidth and capacity.

[0003] Due to this limitation, one existing technique is to deploy large language models with smaller parameters, such as Tongyi Qianwen's 1.8B model, which occupies less than 2GB of memory. However, since larger models exhibit stronger intelligence (a phenomenon known as the "scaling law"), this strategy of deploying "smaller" LLMs is a compromise that reduces the model's intelligence in order to adapt to the memory size limitation, greatly reducing its usability.

[0004] Another solution is based on sparse reasoning. Sparse reasoning uses only some important model parameters for each reasoning task based on the importance distribution of model parameters, which can significantly reduce the cost of reasoning resources while ensuring the quality of reasoning tasks. Although this solution enables the deployment of models with larger parameters on the client side, its design only considers the deployment of homogeneous computing power and ignores the computing power characteristics of heterogeneous hardware in smartphone scenarios.

[0005] In a Chinese patent document with publication number CN117311932A, a scheduling method, device, computer-readable medium and electronic device for model training are disclosed, the method comprising: in the training process of the first model training task, the first microservice corresponding to each training stage of the training process is sequentially run on at least one computing device to perform the operations of each training stage of the training process, and the computing resource type of the computing device used by the first microservice at runtime is the central processing unit computing power or the artificial intelligence processor computing power; when idle computing resources corresponding to the target computing resource type appear on the target computing device in at least one computing device, the second microservice corresponding to the target training stage in the training process of the second model training task is started and run on the target computing device, and the computing resource type used by the second microservice at runtime is the target computing resource type. However, this patent document only realizes fine-grained computing power mining, reduces computing power costs, and does not solve the above problems. Summary of the invention

[0006] In view of the defects in the prior art, the purpose of the present invention is to provide a method, device and system for fast inference of a large language model of a smart phone.

[0007] A large language model fast inference method for a smart phone provided by the present invention comprises:

[0008] Step S1: construct a detection system and use the detection system to collect information about the device under test;

[0009] The detection system includes a large language model;

[0010] The device information to be tested includes hardware configuration information, model configuration information and user requirement information;

[0011] Step S2: enabling the detection system to automatically select a corresponding reasoning strategy according to the information of the device to be tested;

[0012] Step S3: load the parameters of the large language model, initialize the inference environment, and start the inference process;

[0013] The inference process includes a pre-filling phase and a decoding phase;

[0014] Step S4: Generate and return the inference results and form a result report; adjust the inference strategy after obtaining user feedback and optimize the inference process.

[0015] Preferably, step S3 includes the following sub-steps:

[0016] Step S3.1: In the pre-filling phase, the massively parallel matrix multiplication is dispatched to the NPU for calculation, and the weight dequantization and loading are dispatched to the CPU;

[0017] Step S3.2: In the decoding stage, the sparse activation features and inference strategies of the large language model are combined to use the corresponding CPU or GPU for sparse calculations.

[0018] Preferably, the hardware configuration information includes CPU model and core number, GPU model and computing power, NPU model and operating frequency, memory capacity and bandwidth.

[0019] Preferably, step S3 further comprises using a large language model application to preheat and load the parameters into the memory of the device under test.

[0020] Preferably, the step S3.1 includes computing power requirements, memory requirements, and data dependencies of the computing task, and distributes the task to heterogeneous computing units based on these requirements; the heterogeneous computing units include CPU, GPU, and NPU.

[0021] Preferably, step S5 includes using a detection system to record resource usage information during the reasoning process; the resource usage information includes CPU usage, GPU usage, NPU usage, memory occupancy and reasoning time.

[0022] A large language model fast reasoning device for a smart phone provided by the present invention comprises a heterogeneous hardware analysis component, a decision generation component, a heterogeneous computing power mixed computing component, a resource monitoring component and a smart phone device; the smart phone device is equipped with a heterogeneous computing unit including a CPU, a GPU and an NPU; the heterogeneous hardware analysis component is used to collect and analyze the CPU, GPU, NPU, memory configuration information and computing power and power consumption information in the smart phone device; the decision generation component is used to determine the scheduling strategy of different operators in the reasoning process; the resource monitoring component is used to monitor the calculation time of each operator in the reasoning process, the end-to-end reasoning time and the power consumption data.

[0023] A large language model fast inference system for a smart phone provided by the present invention comprises:

[0024] Module M1: Build a detection system and use the detection system to collect information about the device under test;

[0025] The detection system includes a large language model;

[0026] The device information to be tested includes hardware configuration information, model configuration information and user requirement information;

[0027] Module M2: enables the detection system to automatically select the corresponding reasoning strategy according to the information of the device under test;

[0028] Module M3: loads the parameters of the large language model, initializes the inference environment, and starts the inference process;

[0029] The inference process includes a pre-filling phase and a decoding phase;

[0030] Module M4: Generate and return the reasoning results and form a result report; adjust the reasoning strategy after obtaining user feedback and optimize the reasoning process.

[0031] Preferably, the module M3 includes the following submodules:

[0032] Module M3.1: In the pre-filling phase, the massively parallel matrix multiplication is dispatched to the NPU for calculation, and the weight dequantization and loading are dispatched to the CPU;

[0033] Module M3.2: In the decoding stage, the sparse activation features and inference strategies of the large language model are combined, and sparse calculations are performed using the corresponding CPU or GPU.

[0034] Preferably, the hardware configuration information includes CPU model and core number, GPU model and computing power, NPU model and operating frequency, memory capacity and bandwidth.

[0035] Preferably, the module M3 also includes using a large language model application to preheat and load parameters into the memory of the device under test.

[0036] Preferably, the module M3.1 includes computing power requirements, memory requirements, and data dependencies of computing tasks, and distributes tasks to heterogeneous computing units based on these requirements; the heterogeneous computing units include CPU, GPU, and NPU.

[0037] Preferably, the module M5 includes using a detection system to record resource usage information during the reasoning process; the resource usage information includes CPU usage, GPU usage, NPU usage, memory occupancy and reasoning time.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. The present invention can efficiently utilize heterogeneous computing resources: The present invention uses an intelligent scheduling algorithm to fully utilize heterogeneous computing units such as CPU, GPU and NPU in smartphones, thereby optimizing the efficiency of computing resources. Compared with the traditional method that relies on only a single computing unit, the present invention can significantly improve the reasoning speed and performance.

[0040] 2. The present invention enables low-latency reasoning experience: By utilizing NPU for large-scale parallel computing and CPU / GPU for sparse computing at different reasoning stages, the present invention can reduce the computational delay in the reasoning process and provide a faster response time while ensuring computational accuracy.

[0041] 3. The present invention can flexibly adapt to a variety of hardware architectures: the present invention supports a variety of CPUs (such as ARM, RISC-V, x86), GPUs (such as Mali, Adreno, PowerVR) and NPU architectures, and is suitable for smartphones of different brands and models, and has wide applicability.

[0042] 4. The present invention can adapt the optimization strategy: the system can automatically adjust the reasoning optimization strategy according to the hardware configuration of the smartphone and the user's needs (such as latency, memory usage restrictions, and CPU computing power restrictions) to ensure optimal performance under different hardware conditions.

[0043] 5. The present invention is capable of dynamic performance monitoring and optimization: The present invention provides a performance monitoring module that can record resource usage during the reasoning process, generate performance reports, and dynamically adjust execution strategies based on user feedback to continuously optimize reasoning performance.

[0044] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0046] Figure 1 The present invention is a flow chart of the method.

[0047] Figure 2 It is a schematic diagram of the device structure of the present invention. DETAILED DESCRIPTION

[0048] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0049] Reference Figure 1 As shown, the present invention provides a fast reasoning method for a large language model of a smart phone, which utilizes the efficient collaboration of heterogeneous computing units in the mobile phone at different computing stages. First, by systematically analyzing the computing efficiency of different computing resources at different reasoning stages of the large language model, an intelligent scheduling decision plan is formed to make full use of the computing resources such as the CPU, GPU and NPU of the mobile phone to achieve fast reasoning of the large language model.

[0050] Here are the steps:

[0051] Step 1: Systematically analyze the hardware configuration of the mobile phone and automatically select the appropriate reasoning strategy based on the hardware situation, model configuration, and user needs. The process of system detection of the hardware configuration of the mobile phone includes but is not limited to detection of CPU model and core number, GPU model and computing power, NPU model and operating frequency, memory capacity and bandwidth.

[0052] Step 2: The user starts the large language model application, and the application preheats and loads the model parameters into the memory and initializes the inference environment; wherein the process of the application loading the model parameters into the memory and initializing the inference environment includes but is not limited to preheating and loading the model parameters and initializing the inference engine.

[0053] Step 3: In the pre-filling phase, the large-scale parallel matrix multiplication is scheduled to the NPU for calculation, while the weight dequantization and loading are scheduled to the CPU. The process of the intelligent scheduling algorithm analyzing the computing requirements of the inference task includes but is not limited to the computing power requirements, memory requirements, and data dependencies of the computing task, and assigns the task to heterogeneous computing units such as CPU, GPU, and NPU based on these requirements.

[0054] Step 4: In the decoding stage, the sparse activation features of the large language model are combined and sparse computing is performed using a CPU / GPU suitable for flexible computing. The use of CPU and GPU depends on the user's requirements for inference speed and power consumption, which is determined by the inference strategy.

[0055] Step 5: The system records resource usage during the inference process and generates a performance report. Obtain user feedback and adjust execution strategies. The system records resource usage during the inference process, including but not limited to recording CPU, GPU and NPU usage, memory usage, inference time, and finally generates a detailed performance report for users to view.

[0056] Reference Figure 2 As shown, the present invention also provides a large language model fast reasoning device suitable for smart phones, which is composed of a heterogeneous hardware analysis component, a decision generation component, a heterogeneous computing power hybrid computing component, a resource monitoring component and a smart phone device.

[0057] Among them, smartphones are equipped with heterogeneous computing units such as CPU, GPU and NPU. The heterogeneous hardware analysis component is responsible for collecting and analyzing the CPU, GPU, NPU, memory configuration information, computing power and power consumption information in smartphones. The decision generation component is responsible for determining the scheduling strategies of different operators in the reasoning process, and the resource monitoring component is responsible for monitoring the reasoning process; the reasoning process includes but is not limited to the calculation time of each operator, end-to-end reasoning time and power consumption data.

[0058] The present invention makes full use of heterogeneous computing units such as CPU, GPU and NPU in smart phones through intelligent scheduling algorithms, and optimizes the use efficiency of computing resources. Compared with the traditional method that relies only on a single computing unit, the present invention can significantly improve the reasoning speed and performance; by using NPU for large-scale parallel computing and CPU / GPU for sparse computing at different reasoning stages, the present invention can reduce the calculation delay in the reasoning process and provide faster response time while ensuring the calculation accuracy; the present invention supports a variety of CPUs (such as ARM, RISC-V, x86), GPUs (such as Mali, Adreno, PowerVR) and NPU architectures, and is suitable for smart phones of different brands and models, with wide applicability; it can automatically adjust the reasoning optimization strategy according to the hardware configuration of the smart phone and the needs of the user (such as latency, memory usage restrictions, CPU computing power restrictions), ensuring that the best performance can be achieved under different hardware conditions. The present invention provides a performance monitoring module that can record the resource usage during the reasoning process, generate a performance report, and dynamically adjust the execution strategy according to user feedback to continuously optimize the reasoning performance.

[0059] The above is a basic embodiment of the present invention. The technical solution of the present invention is further described below through a preferred embodiment.

[0060] Example 1

[0061] Reference Figure 1 As shown, the specific process of heterogeneous hardware collaborative execution in sparse reasoning of a large language model in the present invention is as follows:

[0062] Step 101: The user downloads and installs the large language model and related inference acceleration components from the official link. The components include pre-trained models, quantized models, optimized inference engines, and hardware acceleration libraries;

[0063] Step 102, the inference system automatically detects the hardware configuration of the mobile phone, including but not limited to detecting the CPU model and core number, GPU model and computing power, NPU model and operating frequency, memory capacity and bandwidth;

[0064] Step 103: Based on the detection results, the system automatically selects a suitable reasoning strategy based on the user's needs (such as latency, memory usage restrictions, and CPU computing power restrictions), such as limiting the number of CPU cores and memory capacity. This strategy aims to make full use of the heterogeneous computing units (CPU, GPU, and NPU) of the mobile phone.

[0065] Step 104, the application loads the model parameters into the memory and initializes the inference environment. The loading process includes preheating loading of the model parameters, memory mapping, and initialization of the inference engine;

[0066] Step 105 , the system allocates tasks to heterogeneous computing units (such as CPU, GPU, and NPU) during the inference process.

[0067] In the pre-filling stage, the inference system is divided into two computing tasks: weight dequantization and matrix multiplication. The system schedules weight dequantization to the CPU for execution. The CPU calls a multi-threaded dequantization operator customized according to the characteristics of vector calculations to convert the weights from a quantization type (such as INT4 type) to a floating-point type that the NPU can perform calculations on. Secondly, the system schedules the matrix multiplication task to the NPU, calls the NPU's matrix multiplication (GEMM) interface (such as the QNN Mulmat API), and uses the NPU's Tensor core for large-scale matrix parallel calculations. After that, the decoding stage begins.

[0068] In the decoding stage, the inference task consists of matrix-vector multiplication calculation tasks. Combined with the sparse activation characteristics of large language models activated by rows / columns, the system schedules the task to the CPU (using AVX vector calculation extension) or GPU (through Vulkan, OpenCL) according to the execution strategy generated in step 103 to perform sparse matrix-vector calculations according to the activation status, skipping unnecessary calculations.

[0069] Step 106, after the reasoning task is completed, the application generates a reasoning result and returns the result to the user;

[0070] Step 107: The system records the resource usage during the inference process and generates a detailed performance report. The recorded content includes but is not limited to the CPU, GPU and NPU usage, memory usage, and inference time. Based on the performance report and user feedback, the system dynamically adjusts the inference strategy to further optimize the inference performance.

[0071] The present invention also provides a large language model fast inference system for a smart phone. The large language model fast inference system for the smart phone can be implemented by executing the process steps of the large language model fast inference method for the smart phone, that is, those skilled in the art can understand the large language model fast inference method for the smart phone as a preferred implementation of the large language model fast inference system for the smart phone.

[0072] Specifically, a large language model fast inference system for a smartphone includes:

[0073] Module M1: Build a detection system and use the detection system to collect information about the device under test;

[0074] The detection system includes a large language model;

[0075] The device information to be tested includes hardware configuration information, model configuration information and user requirement information;

[0076] Module M2: enables the detection system to automatically select the corresponding reasoning strategy according to the information of the device under test;

[0077] Module M3: loads the parameters of the large language model, initializes the inference environment, and starts the inference process;

[0078] The inference process includes a pre-filling phase and a decoding phase;

[0079] Module M4: Generate and return the reasoning results and form a result report; adjust the reasoning strategy after obtaining user feedback and optimize the reasoning process.

[0080] The module M3 includes the following submodules:

[0081] Module M3.1: In the pre-filling phase, the massively parallel matrix multiplication is dispatched to the NPU for calculation, and the weight dequantization and loading are dispatched to the CPU;

[0082] Module M3.2: In the decoding stage, the sparse activation features and inference strategies of the large language model are combined, and sparse calculations are performed using the corresponding CPU or GPU.

[0083] The hardware configuration information includes CPU model and core number, GPU model and computing power, NPU model and operating frequency, memory capacity and bandwidth.

[0084] The module M3 also includes using a large language model application to preheat and load parameters into the memory of the device under test.

[0085] The module M3.1 includes the computing power requirements, memory requirements, and data dependencies of the computing tasks, and allocates the tasks to heterogeneous computing units based on these requirements; the heterogeneous computing units include CPU, GPU and NPU.

[0086] The module M5 includes using a detection system to record resource usage information during the reasoning process; the resource usage information includes CPU usage, GPU usage, NPU usage, memory occupancy and reasoning time.

[0087] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0088] In the description of the present application, it should be understood that the terms "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0089] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A fast inference method for a large language model of a smart phone, characterized in that: include: Step S1: construct a detection system and use the detection system to collect information about the device under test; The detection system includes a large language model; The device information to be tested includes hardware configuration information, model configuration information and user requirement information; Step S2: enabling the detection system to automatically select a corresponding reasoning strategy according to the information of the device to be tested; Step S3: load the parameters of the large language model, initialize the inference environment, and start the inference process; The inference process includes a pre-filling phase and a decoding phase; Step S4: Generate and return the inference results to form a result report; Adjust the reasoning strategy after obtaining user feedback and optimize the reasoning process; The step S3 comprises the following sub-steps: Step S3.1: In the pre-filling phase, the massively parallel matrix multiplication is dispatched to the NPU for calculation, and the weight dequantization and loading are dispatched to the CPU; Step S3.2: In the decoding stage, the sparse activation features and inference strategies of the large language model are combined to use the corresponding CPU or GPU for sparse calculations; The step S3.1 includes computing the computing power requirements, memory requirements, and data dependencies of the computing task, and allocating the task to heterogeneous computing units based on these requirements; the heterogeneous computing units include CPU, GPU, and NPU.

2. The large language model fast inference method for a smart phone according to claim 1, characterized in that: The hardware configuration information includes CPU model and core number, GPU model and computing power, NPU model and operating frequency, memory capacity and bandwidth.

3. The large language model fast inference method for a smart phone according to claim 1, characterized in that: The step S3 also includes using a large language model application to preheat and load the parameters into the memory of the device under test.

4. The large language model fast inference method for a smart phone according to claim 1, characterized in that: The step S4 includes using a detection system to record resource usage information during the reasoning process; the resource usage information includes CPU usage, GPU usage, NPU usage, memory occupancy and reasoning time.

5. A large language model fast inference device for a smart phone, using the large language model fast inference method for a smart phone according to any one of claims 1 to 4, characterized in that: It includes a heterogeneous hardware analysis component, a decision generation component, a heterogeneous computing power hybrid computing component, a resource monitoring component and a smart phone device; the smart phone device is equipped with a heterogeneous computing unit including a CPU, a GPU and an NPU; the heterogeneous hardware analysis component is used to collect and analyze the CPU, GPU, NPU, memory configuration information and computing power and power consumption information in the smart phone device; the decision generation component is used to determine the scheduling strategy of different operators in the reasoning process; the resource monitoring component is used to monitor the calculation time of each operator in the reasoning process, the end-to-end reasoning time and power consumption data.

6. A large language model fast inference system for a smart phone, characterized in that: include: Module M1: Build a detection system and use the detection system to collect information about the device under test; The detection system includes a large language model; The device information to be tested includes hardware configuration information, model configuration information and user requirement information; Module M2: enables the detection system to automatically select the corresponding reasoning strategy according to the information of the device under test; Module M3: loads the parameters of the large language model, initializes the inference environment, and starts the inference process; The inference process includes a pre-filling phase and a decoding phase; Module M4: Generate and return the reasoning results and form a result report; Adjust the reasoning strategy after obtaining user feedback and optimize the reasoning process; The module M3 Includes the following submodules: Module M3.1: In the pre-filling phase, the massively parallel matrix multiplication is dispatched to the NPU for calculation, and the weight dequantization and loading are dispatched to the CPU; Module M3.2: In the decoding stage, the sparse activation features and inference strategies of the large language model are combined, and sparse calculations are performed using the corresponding CPU or GPU.

7. The large language model fast inference system for a smart phone according to claim 6, characterized in that: The hardware configuration information includes CPU model and core number, GPU model and computing power, NPU model and operating frequency, memory capacity and bandwidth.

Citation Information

Patent Citations

  • Model training scheduling method and device, computer readable medium and electronic equipment

    CN117311932A

  • Deep learning reasoning operation method and middleware

    CN111144571A

  • Large language model reasoning optimization method and device, computer equipment and storage medium

    CN117194056A